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Introduction 



Combinatorial counting problems have a long and distinguished history. Apart from their intrin- 
sic interest, they arise naturally from investigations in numerous branches of mathematics and 
natural sciences and have given rise to a rich and beautiful theory. Ranking problems, which 
consist in determining the position of a given element in a well-ordered set, are closely related 
to counting. Random generation problems are less well studied but have a large number of 
computational applications. 

From the structural complexity viewpoint, the study of counting problems was initiated by 



Vahanti (119791) . A parallel approach to random generation problems was proposed by lJerrum. Valiant, and Vazirani 
(Il986h : m particular, they show how the standard reduction from generation to exact count- 



ing can be modified to yield an almost uniform generator giving only approximate counting 
estimates. They also locate the almost uniform generation and approximate counting prob- 
lems for general NP rel ational] within the second level of the (probabi listic) polynom ial time 
hiera rchy (Stockmeve^ 1977). Finally, ranking has been studied by Huynhl ( 1990h and by 
IColdberg and Sipseil (1991) that considered it as a special kind of optimal compression. 



The aim of this thesis is to determine classes of NP relations for which random generation 
and approximate counting problems admit an efficient solution. Since efficient rank implies ef- 
ficient random generation, we first investigate some classes of NP relations admitting efficient 
ranking. On the other hand, there aie situations in which efficient random generation is pos- 
sible even when ranking is computationally infeasible. We introduce the notion of ambiguous 
description as a tool for random generation and approximate counting in such cases and show, 
in particular, some applications to the case of formal languages. Finally, we discuss a limit of 
an heuristic for combinatorial optimization problems based on the random initialization of local 
search algorithms showing that derandomizing such heuristic can be, in some cases, jjP-hard. 
More details follow. 



Ranking. We extend some results about ranking for formal languages to the case of NP re- 
lations, a fact that allows us to introduce two new classes of relations admitting efficient (i.e., 
polynomial time) random uniform generation. In particular, we prove that the classes of NP 
relations accepted by 

(i) unambiguous auxiliary pushdown automata working in polynomial time, and 

'by NP relations here we mean subsets C Z* x Z* such that, for every a, (3 £ Z*, |(3| is polynomially related to 
|a| and the predicate a if P can be decided in time polynomial in [a| (where Z is some finite alphabet). For a more 
detailed discussion, see Section [L2l 



viii 



(ii) nondeterministic Turing machines using s{n) space, i{n) inversions and having ambiguity 
d{n) with s{n) ■ i{n) • d{n) = Oilogn), 



are such that their rank fu nctions can be computed in pol ynomial time. These results follow 
from t he techniques used by lHuynh and, respectively. iBertoni. Mereghetti. and Pighizzinil 

(11994) for the ranking of formal languages. Hence, since we have also proved that efficient rank 
implies efficient random generation, the above classes of relations both admit polynomial time 
random uniform generation. 



Ambiguous descriptions. To deal with the case in which ranking is computationally infeasi- 
ble, we introduce a simple notion of description of a combinatorial structure, together with a 
corresponding notion of ambiguity, and study the problem of uniform random generation and 
approximate counting for structures endowed with such descriptions. We prove a general result 
stating that if a structure S has a description T with polynomially bounded ambiguity and T 
admits a polynomial time uniform random generator (u.rg.), then also S admits a u.r.g. working 
in polynomial time. If, moreover, the counting problem for T is solvable in polynomial time, 
then S admits a fully polynomial time randomized approximation scheme (ra.s.) for its counting 
prob lem. Here, the proofs are based on the Karp-Luby technique fo r sampling from a union of 
sets ( Karp. Luby. and Madras . 1989h and on Hoeffding's inequality ( Hoeffding . 1963 ). a classi- 
cal tool for bounding the tail probability of the sum of independent bounded random variables. 
Such general results can be applied to various classes of languages: 



(i) We show that, for finitely ambiguous context-free languages, a word of length n can be 
generated uniformly at random in 0{n^\ogn) time and 0{n^) space, using 0{n^\o%n) 
random bits. We observe that, in our model of computation, the same bounds for time 
and ra ndom bits are obta ined for the uniform random generation of unambiguous c.f. lan- 
guage (IGoldwurml 1 1 995h . Similar bounds are obtained for the corresponding randomized 
approximation scheme. To prove these results we show in de tail a multiplicity version of 
Barley's algorithm for context-free recognition ^ Barley ( 1970 ). We prove that, for finitely 
ambiguous c.f. languages, the number of derivation trees of an input word of size n can be 
computed in 0{n^\ogn) time and 0{n^) space; 

(ii) We show how to generate, uniformly at random, words from languages accepted by one- 
way nondeterministic auxiliary push-do wn automata \voT\dns, in polynorn i al tirn e and us- 
ing a logarithmic amount of work-space ( Cook . 19701 1971 : Brandenburg . 1977 ). Also in 
this case, we obtain polynomial time u.r.g. and r.a.s. whenever the automaton has a poly- 
nomial number of accepting computations for each input word. Notice that such results 
hold in particular for polynomially ambiguous context-free languages. 

(iii) We consid er the uniform random genera tion and approximate counting of rational tr ace 



languases (iDiekert and Rozenberg| . ll995r) . Binitely ambiguous rational trace languages (IBertoni. Mauri, anc 



19821 : Is akarovitchl . 119871) admit u.r.g. and r.a.s. of the same time complexity of the algo- 



rithms for their recognition problem. Analogously, we obtain polynomial time u.r.g. and 
r.a.s. for the rational trace languages that are polynomially ambiguous. 



Derandomiz ation and com binatorial optimization. We focus our attention on an heuristic 
suggested by iGrossi (I1999I) for improving local search algorithms. The basic idea is to use a 
two phase (randomized) algorithm which in a first phase generates uniformly at random some 
feasible solution and then, starting from the best solution so obtained, performs a local search 



ix 



phase. For a large class of problems, such heuristic is known to give (randomized) approxima- 
tion algorithm with a constant performance ratio. We study in detail the case in which the objec- 
tive function of the combinatorial optimization problem is represented by means of arithmetic 
circuits: this is a quite common situation and applies to a lot of natural problems. Sufficient 
conditions are known in such a case to prove that the first phase of the heuristic can be "de- 
randomized" thus yielding to a deterministic approximation algorithm. On the other hand, by 
investigating a problem of transformation between arithmetic circuits, we bring to light a limit 
of this approach and show that "derandomizing" the first phase of the heuristic is, in some cases, 
a (jP-hard problem. 



Structure of the chapters. 

This thesis is organized as follows. In Chapter [1] we introduce some preliminary notions and 
definitions; in particular, in Section 11.11 we present our model of computation: the PrRAM, 
which is a probabilistic version of the standard random access machine model endowed in ad- 
dition with an unbia sed coin tossing device. In Section [L2l we recall the definition of p-relation 
(iJerrum et al. a fundamental tool to describe formally the problems we deal with in this 



work. Section [T3] concludes the chapter with a short s urvey on t he known re sults about approxi 



mate c ounting and random generation, as discussed by lJerrum et al..(.1986,) : iJerrum and Sinclair 
(Il989h . 



Chapter |2] shows how the cited results on ranking for formal languages can be applied to the 
random generation problem. First of all, in Section lTTl the standard notion of ranking is recalled 
and a proposed extension to the case of relations is given; hence, we prove (Theorem l2.1.1l) that if 
a /7-relation admits efficient (i.e., polynomial time) ranking then it a lso admits efficient r andom 



uniform generation. Hence, in Section 12.21 we extend the result of iBertoni et al.l (119941) about 



the ranking of languages accepted by Turing machines with simultaneous co mplexity bounds 



to /^-relations (Theorem 12.2.11) ; similarly, in Section 123] we extend the result of Huynh ( 1990h 



about the ranking of languages accepted by unambiguous one-way auxiliary pushdown automata 
working in polynomial time to /j-relations (Theorem 12.3. II ). Thanks to Theorem 12. 1.11 these last 
results yield to two new classes of p-relations admitting efficient uniform random generation 
(Corollaries f2TT\ and lZTTT) . 

In Chapter [3l for the sake of simplicity, we turn our attention to combinatorial structures 
introducing a general paradigm that, under suitable hypotheses, leads to polynomial time algo- 
rithms both for random generation and approximate counting problems. We begin by recalling 
some useful definition in Section [TTl adapting in particular to the case of combinatorial struc- 
tures the notion s of uniform r andom generator (u.r.g.) and randomized approximation scheme 
(r.a.s.) given by iJerrum et al. (Il986h for the case of p-relations. In Section 1X2] we introduce a 



simple notion of description for a combinatorial structure together with a related definition of 
ambiguity and prove the existence of a polynomial time u.r.g. (Theorem 13.2.11 ) and of a fully 
polynomial r.a.s. (Theorem 13.2.21 ) for combinatorial structures admitting suitable (ambiguous) 
descriptions. To elucidate such results. Section 13.31 presents some simple application of our 
general paradigm to the case of union and product of combinatorial structures. 

Chapter |4] gives less trivial applications of the results of the previous chapter. In Section |4~T] 
we show a polynomial time u.r.g. and a fully polynomial time r.a.s. for polynomially ambigu- 
ous context-free languages (Theorem 14.1.21 ). Notice that even for finitely ambiguous context- 



X 



free lan guages exact counting is known to be jjPi-compIetqfl (iBertoni. Goldwurm. and Sabadinil . 



1991b!)). Our results depend on a multiplicity version of Barley's algorithm for context-free 
recognition; in particular, for finitely ambiguous c.f. languages, we give an algorithm comput- 
ing the number of derivation trees of an input word of size n in in 0{n^\ogn) time and 0{n^) 
space (Proposition I4.1.1I ). Then, in Section I4.2[ we show that the class of languages accepted 
by polynomially ambiguous one-way auxiliary pushdown automata working in polynomial time 
admits a polynomial time u.r.g. and a fully polynomial time r.a.s. (Theorem 14.2. II) . Finally, in 
Section 1431 we give a polynomial time u.r.g. and a fully polynomial time r.a.s. for polynomi- 
ally ambiguous rational trace languages (Theorem 14.3.11 ): in particular, in the case of finitely 
ambiguous rational trace languages, such algorithms have the same time complexity as the al- 
gorithms for their recognition problem. 

We came back to the more general setting of /^-relations in Chapter [51 where we suggest 
how the results given for the case of combinatorial structures can be generalized to the case of 
/7-relations. More precisely, in Section ISTTl we propose a generalization of Theorem l2. l.ll to the 
case of /^-relations (Corollary 15.1.11 ) which, in particular, leads to a stronger version of Theo- 
rem 12.3.11 this allows one to define a broader class of /j-relations admitting polynomial time 
uniform random generations in terms of relations accepted by polynomially ambiguous one- 
way pushdown automata working in polynomial time (Corollary 15.1.21 ). Sections 15.21 and 15.31 
conclude the chapter by discussing some closure properties with respect to the union and com- 
plement of the classes of p-relations admitting efficient ranking and/or random generation. 

As a very different applications of our results, in Chapter [6l we consider the case of (NP) 
combinatorial optimization problems. After the very short survey of basic notions and defini- 
tions of Section ISTl we f ocus our atten tion to the local search paradigm in Section [6?2l where 



we recall the approach of lGrossil (119991 ) for improving local search algorithms. The determina- 
tion of new classes of /^-relations admitting polynomial time uniform random generators allows 
us to introduce a class of NP optimization problems admitting a polynomial time randomized 
approximation algorithm with a constant performance ratio (Corollary l6.3.1l) . On the other hand, 
in Section 16.41 we show some negative results about "derandomization" and local search. By 
studying a problem of transformation between arithmetic circuits, we show that, in some cases, 
derandomizing can be as hard as solving a ftP-complete problem (Theorems 16.4. II and 16.4.21) . 



^the class JP] is the restriction of )P to functions having unary inputs. 



Per Vamore della bislunga ho tagliato died giovani alberi pioppi e glabri. Di 
tanti die erano ne hofatto dnque cento fogli di carta bianchi e io su quelli ho scritto 
giomi e mesi per fare una storia. Ora che voi la leggete, sapete se vale o non vale 
quei pioppi padani e il tempo, la vita nei mesi, di un uomo. 



Maurizio Maggiani, Mauri mduri 
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Chapter 1 



Preliminary Notions 



Bisognava concludere. Manifestai alia contessina Delrio cid che sentivo di non 
poterle dissimulare piu a lungo. Si rassegnasse all' idea: le diagonali del paralle- 
logrammo si secano nel loro punto mediano. E non e tutto: esse ne dividono I'area 
in quattro triangoli equivalenti. 

Carlo Emilio Gadda, La Madonna deifilosofi 



In this chapter we introduce some preUminary notion and definition. First of all, we discuss 
in some detail the model of computation we refer to in the rest of this work. To this aim, we 
present the RAM model together with a proposed probabilistic extension of such model: the 
PrRAM; we also give an example of algorithm, discussing its implementation on such model in 
order to elucidate some of the concepts here introduced. 

Then, we recall the definition of p-relation and discuss its relevance as a unifying tool for 
the formal definition and analysis of various versions of combinatorial problems, in particular 
with regard to the notion of self-reducibility. 

Finally, we present a brief survey of known results on random generation of p-relations, 
discussing in particular the relationship between (almost) uniform random generation and ap- 
proximate counting. 



1.1 The Model of Computation 

One of the fundamental issues arising in the design and analysis of algorithms and in the inves- 
tigation of the inherent computational difficulty of various problems is the choice of the formal 
model of computation. Even though it is well known that all the "reasonable" computational 
models are polynomially related with respect to the time and space complexity, nonetheless 
each model enjoys some peculiar properties that can help us in the choice. 
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The Turing machine model, for instance, with its primitive instruction repertoire, is suitable 
for the investigation of negative results and lower bounds typically arising in the structural com- 
plexity investigations. On the other hand, random access machines, or the arithmetic machine 
model, lead to a more natural and easily understandable notation for the description and design 
of high level algorithms. 

1.1.1 The classic RAM model 

Since the main aim of this work is to design efficient algorithms rather than to provide lower 
bounds, we focus our attention on the RAM model, that we now briefly recall (for a complete 



refere nce, and for a discussion on the Turing machine model, see (lAho. Hopcroft. and UUman 
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Figure 1.1: A random access machine. 



A random access machine (RAM) consists of a read-only input tape, a write-only output 
tape, a memory, and a program (Figure [TTTI ). The input tape is a sequence of cells each of which 
holds an integer; whenever an integer is read from the input tape, the tape head moves one cell to 
the right. The output tape is a (potentially infinite) sequence of cells that are initially all blank; 
when a write instruction is executed, an integer is printed in the cell of the output tape currently 
under the output tape head, hence the head is moved one cell to the right. The memory consists 
of a sequence of registers mo,mi, . . . each of which is can hold an integer; no upper bound is 
placed on the number of registers that can be used. 

The program is a fixed sequence of (optionally) labelled instructions that are not modi- 
fied during the execution. We assume that there are input-output instructions manipulating the 
tapes (read and write), instructions moving data across memory (load and store), arithmetic 
instructions (add, sub, mult, and div), branching instructions (jump, jzero and jgtz) and, 
finally, halt to stop the computation. Each instruction manipulating data may operate in an in- 
direct addressing mode (for example, for indexing arrays); the operand of such instructions can 
be "=/" to indicate the integer / € Z, "/" to indicate the content of register m,, or to denote 
the content of ruj where j is the content of m,. 
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Semantic 

The semantic of the computation of a RAM can be easily defined with the help of two elements: 
an instruction pointer that determines the next instruction of the program to be executed and 
a memory map M : N — )^ Z that gives, for every / G N, the contents M{i) of the /-th register, 
that is the integer stored in m,. At the beginning of the computation, both tape heads scan the 
respective first (leftmost) cell, the output tape cells are all blank, all the registers are zero (i.e., 
M{i) = for all / G N) and the instruction pointer points to the first instruction of the program. 
During the execution the instruction pointer is modified so that the instructions in the program 
are executed in sequential order unless jump is executed, jzero is executed when M(0) = 0, or 
jgtz is executed when M(0) > 0: in these cases, the instruction pointer is modified according 
to the label that follows the jumping instruction. 



Instruction 


Meaning 


read / 


M{i) takes as value the integer in the current input cell 


read */ 


M{M{i)) takes as value the integer in the current input cell 


write a 


V{a) is written in the current output cell 


load a 


M(0) takes value V{a) 


store / 


M{i) takes value M(0) 


store *i 


M{M{i)) takes value M(0) 


add a 


M(0) takes value M(0) + V{a) 


sub a 


M(0) takes value M(0) - V{a) 


mult a 


M(0) takes value M(0) x V{a) 


div a 


M(0) takes value \M{0)/V{a)\ 


jump b 


the instruction pointer points to b 


jzero b 


if M(0) = 0, then the instruction pointer points to b, else it 




points to the next instruction in the program 


jgtz b 


if M(0) > 0, then the instruction pointer points to b, else it 




points to the next instruction in the program 


halt 


the execution stops 



Table 1. 1: Meaning of RAM instructions. 



The meaning of the various instructions should be clear to anyone who is familiar with some 
kind of assembly language and is briefly recalled in Table 1 1.1 1 where, for the sake of brevity, we 
have denoted by V{a) the value of operand a defined as follows: V{=i) = /, V{i) = M{i) and 
V{*i)=M{M{i)). 

Given the semantic described so far, the computation of a RAM can be essentially seen as 
a (partial) function from the content of its input tape to the (non-blank) cells of its output tape 
(defined whenever the computation halts). More formally, given a RAM P, with an abuse of 
notation, we denote by P also the (partial) function /" : N* — N* such that P{xi,. .. ,Xn) is the 
content of the output tape of P (omitting the trailing blank cells) whenever the computation of 
P starting with xi ...x„ on the input tape halts; otherwise P(xi, . . . ,x„) is undefin ed. The class 
of func t ions s o defined can be related to the cla ss of partial recu rsive functions ( Davis . 19581 : 
Rogersl 1967 ). it is in fact possible to show that (AhoetaL, 1974 ) 



Proposition 1.1.1 The class of (partial) functions that can be computed in the RAM model 
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coincides with the class of recursive (partial) functions. 



Computational complexity 



To define the computational complexity in the RAM model we need to specify the time required 
to execute each instruction and the space used by each register; to this end, two choices are 
usually made up in the literature: the uniform cost criterion and the logarithmic cost criterion 



(lAho et all 119741) . We briefly recall the latter since we shall use it for the analysis of the algo- 
rithms that will be presented in this work. Let /(/) be the logarithmic function on the integers 
defined bjQ 

fLlog(/)J + l if/>0, 
1 if / = 



and define the (logarithmic) cost t{a) of accessing the operand a as t{=i) = l{i), t{i) = l{i) + 
l{M{i)) and t{n) = l{i) + l{M{i)) + l{M{M{i))). Then the cost of the RAM instruction is given 
in Table O 



Instruction Cost 



read / 


/(input) + /(/) 




read */ 


/(input) + /(/) 


+ /(M(/)) 


write a 


t{a) 




load a 


t{a) 




store / 


l{M{0)) + l{i) 




store */ 


/(M(0)) + /(/) 


+ /(M(/)) 


add a 


l{M{0))+t{a] 




sub a 


/(M(0))+f(a: 




mult a 


l{M{0)) + t{a] 




div a 


l{M{0))+tia] 




jump b 


1 




jzero b 


/(M(0)) 




jgtz b 


/(M(0)) 




halt 


1 





Table 1.2: Logarithmic cost of RAM instructions. 



Some remarks 



We conclude this section observing that the choice of the RAM model under logarithmic cost 
criterion enjoys two important properties we look for in the design and analysis of the algorithms 
presented in this work. First of all, the high-level language and structure of the RAM avoids the 
need of the boring specifications usually arising when using the Turing machine model and 
moreover it allows us to design algorithms that are easier to understand and more similar to their 
implementation in today's programming languages. On the other and, by taking into reasonable 
account the size of the operands, the logarithmic cost criterion will allow the analysis of our 



All the logarithms in this thesis are in base 2, unless otherwise specified. 
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algorithms to be realistic in the precise sense of the following proposition. Recall that two 
functions f,g aie said to be polynomially related if two polynomials p,(7 exist such that f {n] < 
p{g{n)) and g{n) < q{f{n)) for every n > 0. Then, it is well known that (AhoetaL, 1974h 



Proposition 1.1.2 The time (and space) cost in the RAM model under logarithmic cost criterion 
and in the (multi-tape) Turing machine model are (respectively) polynomially related. 

This means that every algorithm we design on a RAM can be effectively implemented by some 
fixed hardware so that the logarithmic time required by the algorithm on the RAM is polynomi- 
ally related to the time required by the hardware to actually carry on the computation. 



1.1.2 The PrRAM model 



Since a large part of this work is devoted to randomized algorithms, we need now to introduce 
some probabilistic model of computation. Concerning the source of randomness, two choices are 
usually present in the literature. The first, which characterizes the probabilistic Turing machine 
model (IGillll 19771) . is to enrich the deterministic model of computation with a device able to toss 
an unbiased coin, that is to allow branches during the computation so that each of the two branch 
can be taken with equal probability. The second choice, usually adopted when considering 
models of computation similar to the RAM, is to enrich the deterministic model with one (or 
more) coin of (different) rational, or even arbitrary, bias. It is clear that the two models have 
very different computational power: if one has only a fair coin, then even the simple task of 
choosing with equal probability between three numbers, that is rolling a 3-sided die, has no 
algorithm that always terminates ! 

This asymmetry leads to different approaches in the design of probabilistic algorithms. The 
first possibility is to focus on algorithms working in bounded time , which always terminate 
but that, specially when using the unbiased coin model, can fail to produce their output ex- 
actly according to a desired distribution. If this happens, there are two possibilities: a spe- 
cial symbol can be output denoting the failure of the algorithm, or the algorithm may sim- 
ply try to approximate the desired distribution up to some specified precision degree. On 
the other hand, if one focuses on bounded expected time algorithms, then even in the unbi- 
ased coin model one can look for (possibly very slow, or even nonterminating) algorithms 
whose output exactly follows a desired distribution, or can choose to trade off the computa- 
tion time with the approximation preci sion. Some example s of the bounded time appro ach can 
be found for instance in the works of Jerrum et al. (1986); Jerrum and Sinclair (Il989h and of 

Ipeldman. Impagliazzo. Naor. Nisan. Rudich. and Sharnij ( 1993b. wh i le examples of the expect 

time approach can be found for instance in the works of lAlonsd(ll994b : lBarcucci. Pinzani. and Sprugnoli 
ri992). 

Without going further into details, one has to notice that a bounded time algorithm using 
a special symbol to denote its failure can, in principle, be iterated until success, leading to a 
bounded expected time algorithm; on the other hand, a bounded expected time algorithm whose 
output eventually follows exactly a desired distribution can, in principle, be prematurely halted 
leading to a bounded time approximating algorithm. As one can conclude even from this very 
short presentation, the difference between these two approaches is very subtle and often the 
choice between them is a matter of taste, or opportunity, unless feasibility of the computational 
model is considered. 

As suggested by lJerrum et al. allowing the tossing of an arbitrary (rationally) biased 

coin whose bias depends on the input violates the philosophy of Turing machines, because the 
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coin tossing steps o f different biases cann ot be implemented with a fixed amount of hardware 
and in a fixed time. iFeldman et alj (11993) in fact show that at least two coins of rational bias 
are necessary to simulate a w-sided die, unless n is a. power of two, and that the simulation of 
every coin of rational bias whose denominator is less then n with d coin flips requires at least 
n(logn/logJ) differently biased rational coins. 

Thus, to follow the principle of a feasible model of computation we have stated in the pre- 
vious section, enforced by choosing the logarithmic cost criterion to allow a realistic analysis 
of our algorithms, our choice of the source of randomness will be in favor of the unbiased coin 
model. 



Random tape 



X2 



Input tape 



Program 



Instruction pointer 



Memory 



mo 
mi 



■If Output tape 



Figure 1.2: A PrRAM. 



Our model of probabilistic random access machine (PrRAM) consists of a RAM machine 
endowed in addition with a read-only random tape (Figure 11.21 ) and a further instruction rnd to 
operate on it. The random tape is a sequence of cells each containing or 1 ; when the instruction 
"rnd a" is executed, the random tape head scans V{a) cell and M(0) takes as value the integer 
number given, in binary notation, by the juxtaposition of the read cells. More formally, if R{i) 
denotes the content of the j-th random tape cell rj, when the random tape head is positioned on r, 
and "rnd a" is executed, then M(0) = LJIq' ^ 2^/?(/ + j) and the random tape head is positioned 

on r;+y (a). 



Instruction Meaning 

rnd a M(0) takes value 1^^"' ^^^i^ + J)- 



Table 1.3: Meaning of PrRAM instructions. 
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Semantic 



To define the semantic of a PrRAM computation, we introduce the following notation: let 
P{x\, . . . ,Xn;ri,r2, . . .)be the (partial) function denoting the contents of the output tape (omitting 
the trailing blank cells) at the end of the computation of the PrRAM P having xi ...x^ on the 
input tape and r\r2... on the random tape (whenever the computation halts, otherwise we let 
P{xi , . . . ,x„;r\,r2, . . .) undefined). Since ri , r2, . . . are fixed, the computation can be considered 
deterministic and its semantic can be easily derived by the semantic of the RAM model dis- 
cussed above and from the given meaning of the rnd instruction. Then P{xi,. ..,x„) is defined 
as the random variable P{x\ , . . . ,x^ ;Ri,R2,- ■ ■) where /?i ,/?2, • • • are independent and identically 
distributed Bernoulli random variables such that Pr{/?, = 1} = Pr{^; = 0} = 1/2 for / G N. 



Computational complexity 

Again, to define the computational complexity in the PrRAM model we need to specify the time 
required to execute the rnd instruction; to take into account the dimension of the operand also 
in this case, we also assume that the cost of a "rnd a" instruction is t{a) (where t is defined as in 
the previous section). Moreover, together with the usual time and space complexity measures, 
for the PrRAM model we can also consider explicitly the number of cells scanned on the random 
tape, a quantity which we call the number of random bits used by the computation. 



Instruction 


Cost 


rnd a 


t{a) 



Table 1.4: Cost of PrRAM instructions. 



Some remarks 

We want to highlight again that, thanks to the choice for the source of randomness adopted in 
the PrRAM model discussed in this section, we believe to have attained our purpose to suggest a 
model of computation at the same time feasible and of sufficient high-level to describe in a natu- 
ral fashion the algorithms we will design in this work. As for the RAM case, it is straightforward 
to check that the following holds 

Proposition 1.1.3 The time (and space) cost PrRAM model (under logarithmic cost criterion) 
and the (multi-tape) probabilistic Turing machine model are polynomially related. 



1.1.3 An example of algorithm 



All the algorithms presented in the following will be described us ing a high - level language, 
usually known as "Pidgin ALGOL" (for its informal description, see (lAho et al.Lll974) '): what is 
essential here, is that such algorithms can be translated into a RAM program in a straightforward 
manner, hence we shall never be concerned with the details of this "compilation" process. In our 
Pidgin ALGOL we allow the use of every kind of usual mathematical statement and program- 
ming language construct such as expressions, conditions, statements and (recursive) procedures; 
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moreover, the language has no fixed set of data types: variables can represent integers, strings 
and arrays, or even more complex objects such as sets, lists and graphs. No attempt will be made 
here to give a precise definition of all the constructs used as long as their meaning will be clear 
and their translation into PrRAM code is evident from the context. 

Uniform random generation of integers 

As an example of what stated in this section, we give an algorithm (Algorithm 11.11 ) for the 
uniform random generation of an integer in the range 1 , . . . A^. We discuss its implementation on 
a PrRAM and the consequent analysis of its complexity and correctness. We need to fix some 
< 5 < 1 as an upper bound to the probability that the algorithm fails to give output according 
to the uniform distribution, a fact that the algorithm will signal by use of the special symbol _L 
(we recall that this depends on our choice of an unbiased coin as a source of randomness). 



input N 

r ^ _L, / ^ 

while i < [log(l /5)] and r = ± do 

generate m e { 1 , . . . , 2 ri°g^l } uniformly at random 
iS u <N then r u 
output r. 



Consider first of all the implementation. The "deterministic" statements are all very simple 
to implement on a RAM, as it is easy to verify; in particular, [log 5] is a constant (independent 
of the input), while [logA/^] can be computed in O (log A/^ log log A^) time (see Lemma lA.1.3l for 
details). 

On the other hand, for every m £ N, the "probabilistic" statement "generate m G { 1 , . . . , 2'" } 
uniformly at random", can be implemented on a PrRAM essentially using just a "rnd m" in- 
struction. 

Concerning the correctness of the algorithm, if we denote by R{N) the random variable 
giving the output of the PrRAM R implementing the algorithm, we are able to prove that, for 
every N £N, 



Here, equation (11.11 ) substantially tells that the algorithm gives a "correct" output with probabil- 
ity 1 — 5, and equation (11.21 ) states that, if the algorithm terminates "correctly", then its output is 
actually uniformly distributed over the desired range. The first equation follows by noting that 
the output of the algorithm is ± iff u is greater than A'^ for [log (1/5)] times: 



Algorithm 1.1: a uniform random integer numbers generator. 



Pr{R{N) = _L} < 5, and 
Pr{R{N) = n \ R{N) / 1} = 1 /N, for every « G {1, . . . 



(1.1) 
(1.2) 



Pr{R{N) = ±} = Pt{u > A^}ri°g(V8)l 
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where the first inequality follows from the fact that A'^/2l^'°^''^ > 1/2, for every N > 0. Moreover, 
for every « G {1, . . . ,N}, 

, PT{u = n} 1 f N 1 
Pr{M = n\u<N}- ^ - ' 



Pr{M<A^} 2^iogN] y2^\ogN] J ^■ 

Since the output of the algorithm, whenever it is different from ±, is distributed as u condition- 
ally to the fact that u <N,by the previous equality, we can finally derive equation (ll.ll ). We can 
summarize the result of this section as 

Proposition 1.1.4 For every < 5 < 1, there exists a PrRAM which, on input € N, gives 
in output an integer chosen uniformly at random in {!,..., A'^} with probability 1—5, taking 
O (log (log log A^ - log 5) ) time. 



1.2 The Notion of p-Relation 

We now recall a notion that allows us to formalize several kind of combinatorial problems. The 
use of languages, i.e., subsets of the free monoid r*(where £ is some f inite alphabet), to describe 
formally combinatorial problems is well established (see, for instance, Garey and Johnson . 1979h . 



In a similar fashion, one can also use relations in Z* x £*to describe, with "more richness", such 
problems. As an example, for a suitable encoding, consider the relations 

(1) (f ,x), where F is a boolean formula and T is one of its satisfying truth assignments; 

(2) {G,K), where G is a graph and K is one of its clique^; 

(3) {n,P), where n € N is a positive integer and P is the set of partitions of n; 

(4) {G,K), where K is a. (induced) subgraph!^ of G. 

The use of relations instead of languages is more rich in the sense that they enable us to 
define several "versions" of a combinatorial problem in a natural way. Given some /? C £* x £*, 
define /?(«) = {P G £* : a 7? P}, for every a € £*. Then, for some fixed relation R, one have the 
following kinds of problems 

(a) decision: given a G £*, decide whether there exists some P G R{ol); 

(b) construction: given a € £*, construct, if it exists, some P € /?(«); 

(c) counting: given a G £*, count the number of P G R{a); 

(d) random generation: given a G £*, generate an element of /?(«) uniformly at random. 

Note on passing that, as we will see in Chapter |6l also combinatorial optimization problems can 
be defined in a natural way using relations. 

Such a unifying view of combinatoria l problems by mean s of relations has appeared in the 



hterature in th e form of string relations by lGarey and JohnsonI (1197 9) and of search functions by 



Valiant! (Il978h . It is used in this thesis mainly to relate generation problems, which are the main 



subject of this work, to more familiar combinatorial problems such as existence and counting. 



is a subset of the vertices of G such that every pair of vertices in K are joined by an edge in G. 
is an induced subgraph of G if it is isomorphic to a (proper) subgraph of G. 
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1.2.1 p -Relations 

A class of relations in £* x £* of pa rticular intere s t is th e subclass of the relations that can be 



"checked efficiently"; more formally. iJerrum et all (119861) give the following: 



Definition 1.2.1 A relation /? C Z* x Z* is a /^-relation if 

(i) there exists a polynomial p for which |P| = p{\a\) for every a,P G Z* such that aR (3; 

( ii) the relation R can be decide^ in polynomial time. 

Two remarks are in order. First, we observe that, without loss of generality, for the sake of 



simplicity we fix |p| = /7(|a) while the original definition of Jerrum et al. ( 19861) asks only that 
IPI < Second, here and in the following, by an abuse of notation, for every p-relation, 

by the same letter "p" we will always (implicitly) denote the polynomial whose existence is 
required by the definition. 

Note, for instance, that examples ([T]l and Q above are obviously p-relations, whereas exam- 
ple (O evidently violates condition ^ of Definition 1 1 . 2 . 1 1 and example ^ violates condition dn) 
of the same definition unless P = NP. 

Finally, observe that it is not difficult to reaUze that the above mentioned versions <^ and dcj 
of combinatorial probl ems give rise, in the case of /^-relations, to the well known classes NP, 
of dec ision problems flGarey and Johnson . 1979h and, respectively, (JP, of counting problems 



(IVaUant.. 19791 



1.2.2 Self-reducible p -relations 



We no w recall the definition of self-reducible p-relation as introduced by lSchnon (fl97^ : iJerrum et al 
(Il986h : 



Definition 1.2.2 A p-relation 7? C £* x £* is self-reducible if there exist two polynomial time 
computable functions \|/ : £* x £* ^ £* and a : £* ^ N such that, for every a,(0,^ = bi . . .b„ 

L*, 

(i) G(a) = 0(log|a|), 

(ii) R{a) / implies a{a) > 0, 
(Hi) |\|/(a,(0)[ < [a[ and 

(iv) aR^iff\\f{a,bi...b^^ai))Rba (a)+l • ■■bn- 

Intuitively, a relation is self -reducible if, for every element a of its domain, given a small 
(logarithmic) prefix bi ... ^o(a) of ^ related P, it is possible to efficiently compute a smaller (w.r.t. 
a) element of its domain \\f(a,bi . . .^ci(a)) that is in relation with the suffix ^o(a)+i ■■■bn of p. 

Consider, for instance, the relation of example ([T|l above. Let F be a boolean formula on 
the variables vi,...,v„ and x : {vi,...,v„} {true, false} be a satisfying truth assignment for 
it. By substituting in F the values x(vi), . . . ,x(viog„) in place of the variables vi, . . . ,viogn, one 



that is, the language {aOP : a, p e Z*,ai? P}, where is some symbol not belonging to Z, belongs to P. 
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(efficiently) obtains a new formula F' smaller (with less variables, and hence shorter) than F. It is 
also evident that if Vj , . . . , v'^ j^^^ are the variables of F', then x' : {v[,... , v',_[pg„} — )• {true, false} 
defined as x'(vj) = x(v,+iog„) for 1 < / < n — \ogn is a satisfying truth assignment for F'. Hence, 
such a /^-relation is self-reducible. 



Not every p-relation is self-reducible 



The notion of self-reducibility is ve ry natural an d applies to many of the well known combi 



natorial problems to the extent that iJerrum et all (1198 6) state that "problems which cannot be 



formulated in a self-reducible way seem to be the exception rather than the rule". Nonetheless, 
here we want to note that there are some very natural cases of relations which do not enjoy this 
property and this fact is very relevant for this thesis, as we discuss in the following. 

A natural class of /^-relations can be defined by means of formal languages. For a fixed 
L C Z*, we define the corresponding slice relation Ri as a" Rl a iff a € L" PlL (for some fixed 
aGE). 

Take now as example the regular language cd* of words beginning with a letter c followed 
by any number of <i's. It is very simple to conclude that its corresponding slice relation is not 
self -reducible. For every function a,\|/ (since for every « > it must be o{a") > 0), every 
suffix ...bn of cd* wiU be made only of J's, but no word of this kind belongs to the 

original regular language. Hence, whichever is the length m such that 1"' = \|/(l",Z>i . . .^o(an)), 
condition of Definition ! 1.2.2l can never hold. 



1.3 A Short Survey 



In this section we want to briefly summarize some relationship between uniform random gen- 
eration, approxima te counting and other "versio ns" of combinatorial problems as discussed by 
Jerrum et al.1 (119861 ): iJerrum and Sinclair (Il989h . 



1.3.1 Basic definitions 



First of all, we recall some basic definitions of Jerrum et al. ( 1986^ we here adapt to our notation, 
for the sake of consistency with the following chapter^. We begin with random generation: 



Definition 1.3.1 An algorithm A is an almost uniform generator /or a p-relation R CL* xL* iff, 
for every tolerance 8 € (0, 1) and a € such that R{a) ^ 0, 



(i) A on input a, £ gives output A(a,£) G R{a) U {±}, 

for ever^ p G R{a), {l-e)/#R{a) < Pr{A(a,£) = p | A(a,£) 7^ -L} < {l+e)/#R{a), 
(Hi) Pr{A(a,£) = -L} < 1/4. 

^however, it remains an easy task to verify that the all the definitions we give here, are, to our aim, equivalent to 
the original one. 

^in the following, to avoid confusion with | ■ | which will be used to denote bote the size of an object in a combi- 
natorial structure and the length of a word in a formal language, we denote by #A the cardinality of the set A. 
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Moreover, an almost uniform generator is said fully polynomial ijf it works in time polynomial 
in |a| and log(l/£). 

Observe that the inclusion of the logarithm means that a fully polynomial almost uniform gener- 
ator can, at a modest computational expense, achieve an output distribution which is very close 
to uniform. If we let the tolerance £ = 0, we get the following 

Definition 1.3.2 An algorithm A is an exact uniform generator /or a p-relation /? C Z* x Z* ij^ 

for every a G £* such that R{a) / 0, 

(i) A on input a gives output A (a) € R{a) U {±}, 

(ii) Pr{A(a) = p I A(a) / -L} = I /#R{a), for every p e R{a), and 
(Hi) Pr{A(a,£) = -L} < 1/4. 

We now turn to counting, more prec isely, to approximate counting algorithm, in the sense of 



Stockmeved (ll983h : lJerrum et all (Il986h . 



Definition 1.3.3 An algorithm A is a randomized approximate counter within ratio p : N — > R+ 
for a p-relation /? C £* x £* iff, for every a € T* such that R{oi) ^ 0, 

(i) A on input a gives output A(a) G Q U {J-}, 

(ii) Pr{l/p([a[) <A(a)/#/?(a) <p(|al) \ A{a) ^ ±} > 3/4 and 
(Hi) Pr{A(a) = ±} < 1/4. 

Definition 1.3.4 An algorithm A is a randomized approximation scheme /or a p-relation R C 
Z* X L* iff, for every a G E* such that R{a) / and every £ G (0, 1), 

(i) A on input a,£ gives output A(a,£) G Q U {^}, 

(ii) Pr{(l-£)#/?(a) <A(a,£) < {l+e)#R{a) \ A{a,e) ^ ±} > 3/4 and 
(Hi) Pr{A(a,£) = -L} < 1/4. 



Moreover, a randomized approximation scheme is said to be fully polynomial time whenever it 
works in time polynomial in n and l/£. 



We conclude this section noting that the constants 1/4 and 3/4 present in the definitions of 
(almost) uniform generator, randomized approximate counter and approximation scheme can be 
replaced by other suitably chosen constants, leaving such definitions substantially unchanged by 
essentially the same arguments discussed in Section [3TT] 
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1.3.2 A theoretical solution to random generation 

A widely discussed topic in the literature is the strict relationship existing between the problem 
of counting and genera ting combinatorial objects. This relationship is made precise in the work 
of Jerrum et al. (Il986h by means of orac le (probabilistic) Turing machines with oracles in the 



polynomial hierarchy (IS tockmey en. 1 1 977h . 



Here, we want only to briefly recall that a Turing machine with oracle O C £* is a Turing 
machin^ endowed in addition with an oracle tape and a distinguished query state. The com- 
putation of such machine can be described as for the standard Turing machine except for the 
fact that when the machine enters the query state, with the string co written on the oracle tape, 
the "oracle" answers the question "co S O" by replacing, in one single move, the contents of the 
oracle tape with (a suitable encoding of) "yes/no". The polynomial hierarchy of Stockmeyer is 
defin ed building a hierarchy of classes of languages by means of such oracle Turing machines 



(see (IStockmeyeii . 119771) for more details and for a definition of the classes of formal languages 
IP). 

n' 



For the case of exact uniform generation, Ijerrum et alJ (Il986h prove the following 



Proposition 1.3.1 Let R QT,* xT,* be a p-relation. Then there exists a polynomial time oracle 
(probabilistic) Turing machine that is an exact uniform generators for R either taking a suitable 
oracle in j^P, or in Zj- 

In the ca se of almost uniform generation, a weaker request for the oracle is possible; again 



Jerrum et all (11986') show the following 



Proposition 1.3.2 Let R C Z* x Z* &e a p-relation. Then there exists an oracle (probabilistic) 
Turing machine, with a suitable oracle in NP fZ^j, which is an almost uniform generator for R 
working in time |a| and l/e. 

Observe that a usual computational complexity co njecture is that the classes NP , HP an d Z2 
contain highly intractable problems (see, for instance Balcazar. Diaz, and Gabarrd 1995); for 



this reason, the above mentioned results are to be considered mainly of theoretical interest. 



1.3.3 Relationships with other versions of combinatorial problems 

In this subsection we discuss the relationships of the random generation problem with other 
versions of combinatorial problems: namely the decision and counting versions. 

It is clear that a procedure for counting the elements of R{oC) must in particular decide if 
7? (a) = 0, and hence counting is computationally at least as hard as decision. The first evidence 
that the counti ng version ca n be harder than the decision one for significant natural problems 
was proven by Valiant (Il979) . Consider the /^-relation {G,M) where G = {V,E) is a graph and 



M C £ is a perfect matching of G; Valiant showed that counting the number of perfect matching 
in a bipartite graph (or, equivalently, to compute the permanent of a {0, 1} valued matrix) is 
(jP-complete and hence likely to be computationally intractable, whereas deciding whether a 



^see Section [Z2l for further details on Turing machines, and lGarev and JohnsonI ( Il979h for a formal definition of 
the oracle model. 

^a perfect matching of a graph is a set of edges such that every node of the graph is the endpoint of precisely one 



edge in the match. 
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bipartite graph contains a perfect matching (or, equivalently, deciding if the permanent of a 
{0, 1} valued matrix is non-zero) is in P, by virtue of the classical "augmenting path" algorithm. 

Now we give some evidence that uniform generation is, from the computational complexity 
point of view, somewhere in between decision and counting. 

As an example of the fact that the uniform generation version of a problem can be harder than 
the construction (and hence decision) one, consider the p-relation Ri = {G,c) where G = {V,E) 
is a directed graph and c is a directed simple cycle of G. As one can verify, the decision version 
of the combinatorial problem given by the relation Ri is easily solvable; even the construction 
vers ion: given a graph G, to output one of its (directed, simple) cycles, is clearly in P. Nonethe- 



less, IJerrum et al.. (J98d) proved the following 



Proposition 1.3.3 If there exists a polynomial time uniform generator for R\, then NP = RP. 

Hence, as an example of the fact that the uniform generation version can be easier than the 
counting one, consider the jo-relation R2 = {F,x) where F is a boolean formula in disjunctive 



normal form (DNF), and x is one of its satisfying assignments. One can verify (IJerrum et al.L 



1986h that the counting problem for R2 is t]P-complete, a fact that comes from a reduction from 



the analogous problem for conjuncti ve normal for m (CNF) boo l ean fo rmulas which has been 
proven (jP-complete by Simon ( 1977b . Nonetheless, Jerrum et al. ( 19861) prove the following 



Proposition 1.3.4 There exists a polynomial time uniform generator for R2. 



1.3.4 Almost uniform generation and approximate counting 

Going back to almost uniform generation, one finds again a strong relationship with approximate 
counting. Let /? C £* x Z* be a self-reducible /^-relation and let \|/ and a be the functions cited 
in Definition 1 1.2. 2 1 F or some a £ £* belonging to the domain of R, define the tree of derivations 



(for more details, see I Jerrum and Sinclain.ll989i) of R as the tree whose vertices are pairs 



v = (5,y) gS* xE* such that 

(a) the root is (a,£), where £ is the empty word; 

(b) for any other node v = (8,7) 

- if |6| = 1, then v is a leaf, otherwise 

- if |8| > 1, then V has a child Vco = (\|/(5,(o),Y-(o) for every CO gZ''^^) suchthat/?(\j/(8,a))) 7^ 
0. 

It should be clear that the second part of all the vertices are distinct and that the second part of 
the leaves are precisely the elements of R{a), without repetitions. The bounds on \|/ and a in the 
definition of self-reducibility ensure that the depth of r«(a) and the number of children of every 
vertex of Ti{{a) are bounded by a polynomial in |a|. 

Most k nown uniform generation a lgorithms for combinatorial structures (see, for example 



the book of iNijenhuis and Willi (Il978h . or even Algorithm 14. II of Section 14.1.11 ) may be viewed 
as instances of the following reduction to the corresponding counting problem. Given that the 
structures are described by a self-reducible relation R, select a random path from the root of 
Tr to a leaf, at each stage choosing the next edge with probability proportional to the number 
of solutions in the maximal subtree rooted at its lower end. This information may be obtained 
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from a counter which evaluates the function #/?(co) for appropriate (O in the tree. Moreover, 
by appending a correction process (based on the a posteriori probability of the path), such a 
procedure can be made to work even if the counter is slightly inaccurate. 

More formally, let R C L * x £* be a self-red ucible p-relation such that |P| = 0(|a|'^'^) if 



a /? P, for some Kr > 0. Then, IJerrum et al.l (119861 ) have proven that 



(i) if there exists a polynomial time randomized approximate counter for R within ratio 1 + 
|a|'^'^, then there exists a fully polynomial time almost uniform generator for R; 

(ii) if there exists a polynomial time almost uniform generator for R with tolerance |ap'^'', then 
there exists a fully polynomial time randomized approximate counter for R. 

This, in particular, implies that the following holds 

Proposition 1.3.5 LetR QT,* xT,* be a self-reducible p-relation. Then there exists a fully poly- 
nomial time randomized approximate counter for R iff there exists a fully polynomial time almost 
uniform generator for R. 



On the other hand, when a rather cruder counting information is available (to within a con- 
stant factor, say) the above mentioned scheme of random generation through (approximate) 
counting breaks down owing to the accumulation of errors which are too large to be corrected. In 
such a case a more flexible and self-correcting approach is taken by iJerrum and Sinclair (Il989h 



in which a random process moves dynamically around the tree Tr, with backtracking allowed. 
The generator hence views the vertices of the tree as the states of a Markov chain in which there 
is a non-zero transition probability between two states iff they are adjacent in Tr. The transition 
probability themselves are computed with the aid of the crude approximate counter. Clearly, 
all states communicate, so that, leaving aside questions of periodicity, if the chain is allowed to 
evolve for t steps (from any initial state), the distribution of its final state approaches a unique 
stationary distribution as f — > oo. If the transition probabilities are such that the stationary distri- 
bution is uniform over the leaves of Tr, then one gets an almost uniform generator by simulating 
the Markov chain for sufficiently many steps. 

The study of the rapid mixing property of such a chain, i.e., the speed of the c onvergence of 
the chain to the stationary distribution, is one of the main result of Jerrum and Sinclair (1989I) . 



Thanks to a very sophisticated analysis, the authors are able to prove that, if /? C E* x E* is a 
self-reducible /^-relation for which there exists a polynomial randomized approximate counter 
within ratio 1 + 0{\ql\^) for some arbitrary K € R, then 

(i) there exists a fully polynomial almost uniform generator for R, and 

(ii) there exists a fully polynomial approximation scheme for R. 

This fact, in particular, proves that approximation counting problems, in the case of p- 
relation, cannot give rise to an "approximation hierarchy" of relations (as, for instance, in the 
case of combinatorial optimization problems, as discussed in Section [6. 1.21 ). From the previous 
results, in fact, one can obtain the following surprising 

Proposition 1.3.6 Let R T,* x T,* be a self-reducible p-relation. Let A C R such that there 
exists a polynomial randomized approximate counter within ratio 1 + 0(|a|'^) for some K S A. 
Then, either A = 0, or A = R. 



Chapter 2 



Uniform Random Generation 
Through Ranking 



Cost si potesse dimezzare ogni cosa intera, . . . cost ognuno potesse uscire dalla 
sua ottusa e ignorante interezza. . . Se mai diventerai metd di te stesso, e te I'auguro, 
ragazzo, capirai cose al di la della comune intelligenza dei cervelli interi. Avrai 
perso metd di te e del mondo, ma la metd rimasta sard mille volte piU profonda e 
preziosa. 

Italo Calvino, // visconte dimezzato 



In this chapter we discuss how to apply the concept of ranking to the problem of uniform 
random generation of p-relations. To this end, we briefly recall the concept of ranking for 
formal languages and suggest an extension of such concept to the case of relations. Then we 
prove a theorem stating that if a /^-relation is rankable in polynomial time, then it admits a 
polynomial time uniform random generator. Finally, we extend some known results on ranking 
for formal languages to the case of /j-relations, a fact that allows us to introduce two new classes 
of polynomial time rankable /^-relations. 



2.1 Ranking p-Relations and Uniform Random Generation 

Suppose that, given a totally ordered set of objects, we are able to tell how many they are and 
also to determine each of them once its position in the order is given. Then, to generate such 
objects uniformly at random, we can simply pick an integer uniformly at random between 1 and 
their total number, returning then the corresponding object. We now show how this argument 
can be made precise in the case of /^-relations. 
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2.1.1 The ranking for formal languages 

Suppose that ^ is a total order relation over some finite alphabet £, then this order can be 
extended to a total order relation ^lex over the elements of £*, called lexicographic order: if 
(0,(o' G £*, then CO ^lex «>' iff either ©a = co', or co = yaa, co' = jba' with a and a^b 
(where Y,CT,a' € £* and a,b ^ £). Hence, we have the following 

Definition 2.1.1 The rank ri of a formal language L is the function r/, : £* ^ N defined as 
rtiio) = #{(o' : co' e L, [co'l < [co| or |co'| = |co| and co' ^lex o}/ conversely, the unrank ul is a 
function : N — ?> Z* such that ui[n) = CO, where CO satisfies ri{(i)) = n and for every co' ^lex K>. 
rL{(o') < n (here we assume \L\ = ooj. 



The complexity of computing the rank function has been studied in (IGoldberg and Sipser . 



19911) where it has been considered as a special kind of optimal compression: ranking is in 
fact related to the more general problem of storing and retrieving strings efficiently, a task very 
similar to the one we sketched at the beginning of this section. As one can verify, the ranking 
of every language in P belongs to (jP; in particular, several languages in P are very hard to 
rank. This is the case, for example, of languages accepted by nondeterministic log-time bounded 
Turing machines, log-space bounded deterministic Turing machines, one-way nondeterministic 
log-space bounded Turing machines, uniform families of constant depth and polynomial size 
unbounded fan-in circuits, CRCW P-RAM working in constant time with a polynomial number 
of processor, two-way deterministic checking stack automata, two-way deterministic pushdown 
automata and one-way two-head deterministic finite state automata. For all these classes of 
languages, as proved by Huynh (1990 ). the ranking is even flP-hard. 

Nonetheless, classes of languages for which it is possible to efficiently compute the rank 
function are known. For regular languages, the r anking is NC' -reducible to integer division 
( Bertoni. Bruschi. and GoldwurmL 1991a : HuynhL 1991 ). and hence it can be s olved by log - 
space uniform boolean circuits of 0(log?iloglog?i) depth and polynomial size (ICookL Il985h . 
Moreover, for every language accepted by one-way unambiguous log-space bou nded Turing ma- 
chines, the rank function belongs to the class DET (IGoldberg and Sipseii.ll99ll) of all functions 
NC' -reducible to computing the determinant of an integer matrix (ICookL Il985h : this result has 
been furth er extended to one-way log-sp ace bounded Turing machines with bounded ambiguity 
degree by iBertoni and Goldwunn I (^1993^, and similar resul ts for Tur ing machines with simulta- 
neous complexity bounds are given by Be rtoni et all (11994) . Finally. iHuynhl (1 199(1) showed that 
the rank function is in NC^ for all languages accepted in polynomial time by one-way unambigu- 
ous (log-space) auxiliary pushdown automata hence, in particular, for unambiguous context-free 
languages. 



2.1.2 The case of relations 

We now extend the above-mentioned definition of ranking, usually given for formal languages, 
to the case of relations. 

Definition 2.1.2 The rank of a relation /? C £* x Z* is the function : Z* x Z* ^ N defined 
as rs(a, P) = : a /? P', < |P| or |P'| = |P| and p' ^lex P}.' conversely, the unrank ur is 
the function : £* x N — )• £* such that ur (a, n) = p, where p satisfies r^ (a, P) = « and for every 
P' ^LEx p. P') < n whenever n < #{P : a R P}, else M«(a,«) is undefined. 
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Observe that the notion of rank for languages and the one here given for relations are strictly 
related. Given a relation RCZ* x £*, define the "corresponding" language as 

LR = {aOp:a/?P}, 

where is some distinct symbol not in £ such that a =^ for every a S £; conversely, given a 
language L C £*, define the "corresponding" slice relatioiQ as 

a/?Lp iff a = G Lnl",?i G N, 

for some a GL.lt is then possible to check that if /? is a /^-relation, the following equalities hold 





= r«,(Ol"l a) + 'l:V«,(0',0'), 

i=l 

= rL(a) -maxrL(Y), 

|y <" 


(2.1) 




(2.2) 


P) 


= ri,(aOP)- max ^.(yOO^^I^I^), 
y^LExoi 


(2.3) 




= r«(a,P)+ X rR{y,^P^\y\^). 


(2.4) 



In particular, it is possible to prove that 

Lemma 2.1.1 If R <^L* xL* is a p-relation, then ri G FP ijf r^^ G FP and if r^^ G FP, then 
rR G FP. 

Proof . The proof follows from equations (I2.1I )- (I2.3I) . since the right hand side, in the given 
hypotheses, can be computed in polynomial time. Observe that, for equation (I2.4I ). the sum 
extends over an exponential number of items, so the lemma holds in the stated direction. I 

Given these simple relationships, one can in principle directly apply the known result about the 
ranking for formal languages to the case of relations. Nonetheless, in the next two sections, we 
will give some further independent result, strictly concerning the case of p-relations. 

2.1.3 The unranking 

If the ranking is easy for some language L, in general it is not also true that the unranking is 
similarly easy. Let, for instance, L be the unary language {a^" : n G N}: it is evident that even 
if the ranking amounts simply to the computation of the a logarithm in base 2, the unranking 
function (even if its input is encoded on a unary alphabet) has an exponentially long output, thus 
it cannot at all be computed in polynomial time! 

The situation, with /^-relations, is different, since the codomain of such a relation, for every 
element of the domain, has a cardinality that is bonded above by a function the size of the 
element itself. Hence 

Lemma 2.1.2 IfR C £* x £* is a p-relation and r« G FP, then ur G FP. 

Proof. Since #{p : a/?p} < 2'^(l«l) , ul can be computed by binary search with at most 0{p{\(x\)) 
calls to rR. I 

' see Section [L2l 
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2.1.4 A uniform random generator 

We are now able to state formally the claim sketched at the beginning of this section. Recall 
that in Section 11.1.31 we have designed an algorithm on PrRAM able to generate (with high 
probability) a number in {1, . . . ,N} uniformly at random in time O (log A/^ log log A/^) (see Propo- 
sition [TTL41). It is straightforward then to conclude that 

Theorem 2.1.1 If R QT,* xT,* is a p-relation and S FP, then R admits a polynomial time 
uniform random generator 



In the next two sections of this chapter we show two classes of p-relations, characterized in 
terms of acceptor devices, that admit "efficient" (i.e., polynomial time) ranking. Thanks to the 
last theorem of this section, for both such classes it will indeed be possible to design "efficient" 
uniform random generators. 



2.2 Turing Machines witli Simultaneous Complexity Bounds 

In this section, we show how it is possible to extend to the case of /^-relations the result about the 
ranking of languages acce pted by nondeterministic Turing machines with simultaneous bounds 
on their resources of iBertoni et al. (1994). 



2.2.1 The case of formal languages 



We begin by briefly recal ling the model of Tur ing rn achine we refer to in this section (for a more 
detailed description, see ( Garey and Johnson, 19791) ). 
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Figure 2.1: A Turing machine. 



A (deterministic) Turing machine (TM) consists of a read-only input tape, a read-write out- 
put tape and & finite control (Figure |2?T]) . The tapes are sequences of cells each of which can hold 
a tape symbol of some alphabet £; the finite control can be in one of a finite set of states, two of 
which are distinguished and called the initial state and accepting state. Given the symbols un- 
der the input and work tape heads and the current state a next-move function determines the next 
state, the head movements on both tapes and possibly a symbol to be written on the work tape. 
The semantic of the computation of the machine on input CO = cJi . . . a„ can be defined as follows: 



§2.2 



Turing Machines witii Simultaneous Complexity Bounds 



21 



initially the input tape contains ai . . . a„ all work tape cells are blank and both tape heads scan 
the first (leftmost) cell of the tapes. The input co is said to be accepted iff the TM, started in 
the initial state, makes a sequence of moves which eventually enters the accepting state. The 
language accepted by a TM is the set accepted inputs. 

A nondeterministic Turing machine (NTM) is simply a Turing machine in which the next- 
move function, instead of being a (partial) map from the tuple "state, input tape symbol, work 
tape symbol" to the tuple "state, input tape move, work tape move and symbol to write" as 
in the deterministic case, becomes a map from the same domain to subsets of the codomain. 
The computation can then take "nondeterministically", at every step, one of the possible next 
moves given by such mapping. In this case a string is said to be accepted if at least one of such 
nondeterministic computations ends in an accepting state. 

Various computational resources for such models can be defined as follows, according to 
the usual worst case criterion; observe that we assume without loss of generality that all the 
computations eventually halt. We say that a (deterministic) Turing machine works in time t{n) 
(respectively, uses space s{n), or makes i{n) inversions) iff, for every input ai, . . . ,a„ G £, the 
computation of the machine makes at most t{n) moves (respectively, consumes at most s{n) 
cells of the work tape, or has the input tape head change its direction at most i{n) times). For the 
nondeterministic case, the time, space and inversion resources are similarly defined by taking 
the smallest amount of each resource consumed on every input by one of the possible nonde- 
terministic computations; moreover, the ambiguity d{n) is defined as the maximum number of 
accepting computations, taken over all the inputs ai , . . . , a„ € £. 



We can now define a class of languages accepted by such machines with simultaneous 
bounds on the computation: 

Definition 2.2.1 A language L C £* belongs to ^{s{n),i{n),d{n)), whenever it is accepted by 
a nondeterministic Turing machine that simultaneously uses space s{n), makes i{n) inversions 
and has ambiguity d{n). 



Finally, we are able to formally state the following result obtained by lBertoni et al.l (119941) . 



Proposition 2.2.1 If a language L C £* belongs to ^{s{n),i(n),d(n)) and s{n) ■ i{n) ■ d{n) = 
0{\ogn), then ri € FP. 



2.2.2 An extension to p -relations 

In order to apply the previous result to the case of /^-relations, first of all we need to extend the 
model of computation. The more natural way of doing so, as depicted in Figure l2!2l is to add to 
the standard Turing machine an additional (two-way read-only) input tape and to reserve each 
of the two input tapes to, respectively, the domain and codomain part of the relation. 

The semantic of the computation of the extended Turing machine on input (a, P) can be 
defined as follows: initially the domain input tape contains a = ai . . .a„, the codomain input 
tape contains P = . . .Z?„j, all work tape cells are blank and both the tape heads scan the first 
(leftmost) cell of the tapes. The input (a, P) is said to be accepted iff the extended TM, started 
in the initial state, makes a sequence of moves which eventually enters the accepting state. The 
relation accepted by an extended TM is the set of ordered pairs of strings of input symbols so 
accepted. The semantic for the nondeterministic version can be defined similarly. 

The resources of time, space and ambiguity for the extended model are defined as before, 
but the number of input tape inversions are counted only for the codomain input tape head (the 
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Figure 2.2: An extended Turing machine for recognizing relations. 



domain input tape head has always an unrestricted number of moves). So we can finally extend 
Definition l2.2.1l to the case of relations, with a little abuse of notation. 

Definition 2.2.2 A relation /? C Z* x Z* belongs to ^(s{n) ,i{n) ,d{n)), whenever it is accepted 
by an extended nondeterministic Turing machine which simultaneously uses space s(n), makes 
i{n) inversions and has ambiguity d{n). 

We can fina l ly giv e our first result on "efficiently" rankable relations. By adapting a proof 



of iBertoni et all (11994) . we can indeed state the following 



Theorem 2.2.1 If a p-relation /? C Z* x Z* belongs to ^{s{n),i{n),d{n)) and s{n) ■i{n) ■d{n) = 
Oilogn), then r^ S FR 

Proof . Note that every TM accepting a language in ^{s{n),i{n),d{n)) can be efficiently simu- 
lated by one-way (i.e., with no input tape head inversions) TM using space 0{s{n) ■i{n)) of the 
same order of ambiguity dBertoni et al.l il994h . It is then easy to check that the same holds for 
the extended model. 

Moreover, if M is an extended TM accepting a p-relation R, for every a G Z* and (3 G 
ZP(I"I) it is immediate to construct a one-way extended TM M^^\ of the same order of space and 
ambiguity, accepting the /^-relation R^^^ satisfying a R^^^ y iff a /? y and y ^lex P- 

Hence, to prove the statement of the theorem, it is enough to show that if /? is a /^-relation 
accepted by some one-way extended TM using space s{n) and having ambiguity d{n) such that 
s{n) ■ d{n) = Oilogn), then the cardinality of {y : a R^^^ y} = rs(oc, P) can be computed in a time 
polynomial in [a|. 

It is not hard to see that rR(a, (3) is exactly the number of y G Z^^'"'^ for which M^^\ having 
a on the first input tape and y on the second, halts in an accepting state. 

Since both input tapes are read-only, for every a G Z", p G Z''^") and « G N, the configurations 
of M'P) having a on the first input tape are at most 2'^('("^' = n^^^\ Then (for instance by 
simulating M^P) with a on the first input tape) one can efficiently build two matrices Cq" '^', for 
a G Z, of polynomial size and values in Z, such that the (/,7)th entry of ci" '^'' is 1 iff M^P) moves 
from the /th to 7th configuration having a on the first input tape and reading a on the second 
input tape. 
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If 71 and r\ are, respectively, the characteristic vector of then initial and accepting configura- 

(a,p)„(a,p) 



tions of M'P', then one can verify that tiQ" Ci 



C2 



■ Ccpjij'j'ri < d{n) is the number of accepting 
computations of M^P' on input {(X,y) where y= c\C2 ■ • ■Cp(^„y Here we have implicitly assumed 
that M^P) always moves the head o n its second input tape , but t his is not a restriction since 
stationary moves can be eliminated (IBertoni and Goldwurml. Il993h by computing the transitive 
closure of suitable transition matrices representing stationary moves 



Let now q be an integer polynomial of degree d{n) such that ^(0) 
I < £ < d{n); then, it is possible to check that 



and q{l) = \ for 



p(n) 



(a,p)„(a,p) 



i— 1 C\ ,C2, 



The above computation is esse ntially the same performed as the final step of Proposition 3 in 
(IBertoni and Goldwurml 1 1993h and, as shown in the same paper, it can be transformed, using the 
matrix algebra given by direct sum and Kronecker product, in a form suitable to be computed 
by boolean circuits of polynomial size and depth 0(log"?i), hence, in overall polynomial time. I 
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From this result, by immediate application of Theorem l2. 1.1[ we can introduce, via extended 
Turing machines with simultaneous complexity bounds, a new class of /j-relations admitting a 
polynomial time uniform random generation 

Corollary 2.2.1 If a p-relation /? C E* x E* belongs to I^{s{n),i(n),d{n)) and s[n) ■ i{n) ■d{n) = 
0(\ogn), then it admits a polynomial time uniform random generator 



2.3 One-way Auxiliary Pushdown Automata 

In this section, we show how to extend to the case of p— relation the result about the ranking of 
languages accepted by auxiliary pushdown automata by .Huynh (199ft') . 



2.3.1 The case of formal languages 



Also in this case, we begin by briefly recalling the model of auxiliary pushdown autom ata we 
refer to in this section (for a more detailed description, see (IHopcroft and Ullmanl.ll979l') '). 
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Figure 2.3: An auxiliary pushdown automaton. 



A (deterministic) one-way auxiliary pushdown automaton (I- AuxFDA) consists of a one- 
way read-only input tape, a two-way read- write auxiliary tapda a pushdown store and a finite 
control (Figure 12.31 ). The tapes and the store are sequences of cells each of which can hold a 
symbol of some alphabet £; one of these symbols is distinguished and called the initial pushdown 
symbol. The finite control can be in one of a finite set of states, two of which are distinguished 
and called the initial state and accepting state. Given the symbols under the input and auxiliary 
tape heads, the symbol on top of the pushdown store and the current state, a next-move function 
determines the next state, the auxiliary tape head movement with possibly a symbol to be written 
on it and an action regarding the store which can either be a push of a symbol on top, or a pop of 
a symbol from the top of the store. The semantic of the computation of the automaton on input 
CO = . . .£?„ can be defined as follows: initially the input tape contains ai ...an, the pushdown 
store contains the initial symbol and all auxiliary tape cells are blank and both tape heads scan 
the first (leftmost) cell of the tapes. The input co is said to be accepted iff the automaton, started 
in the initial state, makes a sequence of moves which eventually enters the accepting state, with 
the work tape empty and the pushdown store containing only the initial symbol. The language 
accepted by such an automaton is the set of accepted strings. 

^as will be stated in the following, the auxiliary tape has a logarithmic bound on the number of usable cells. 
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A nondeterministic one-way auxiliary pushdown automaton (1-NAuxPDA) is simply a one- 
way auxiliary pushdown automaton in which the next-move function, instead of being a (partial) 
map from the tuple "state, input tape symbol, work tape symbol and top of the pushdown store 
symbol" to the tuple "state, auxiliary tape move and symbol to write on, action on the pushdown 
store" as in the deterministic case, becomes a map from the same domain to subsets of the 
codomain. The computation can then take "nondeterministically", at every step, one of the 
possible next-moves given by such mapping. In this case a string is said to be accepted if at least 
one of such nondeterministic computations ends in an accepting state. 

Similarly to what happens with TM, various computational resources for such models can 
be defined, according to the usual worst case criterion; observe that we assume without loss 
of generality that all the computations eventually halt. We say that a (deterministic) one-way 
auxiliary pushdown automaton works in time t{n) (respectively, uses space s{n)) iff, for every 
input a\. .Mn G £*, the computation of the automaton makes at most t{n) moves (respectively, 
consumes at most s{n) cells of the auxiliary tape). For the nondeterministic case, the time and 
space resources are similarly defined by taking the smallest amount of each resource consumed 
on every input by one of the possible nondeterministic computations; moreover, the ambiguity 
d{n) is defined as the maximum number of accepting computations, taken over all the inputs 
ai . . .a„ € r*. 

Once we have defined the space resource, which measures only the space on the auxiliary 
tape, not of the pushdown store, we restrict the above definitions of 1-AuxPDA and 1-NAuxPDA 
to automata which use at most logarithmic spac^ i.e., for which s{n) = 0{\ogn). Moreover, the 
one-way unambiguous auxiliary pushdown automata (1-UAuxPDA) is a 1-NAuxPDA for which 
d{n) = 1, i.e., an automaton which has at most one accepting computation for every accepted 
string. 



Finally, we can formally state the following result of Huynhl ( 1990 ) 



Proposition 2.3.1 If a language LOL* is accepted by a polynomial time one-way unambiguous 
auxiliary pushdown automaton, then ri G FP. 

Notice that the class of languages accepted by polynomial time one-way nondeterministic 
auxiliary pushdown automata is exactly the class of languages that are reducible to context-free 
languages via one-way log-space reductions (IBuntrock and Lorysl.ll992h . 



2.3.2 An extension to p -relations 

Again, in order to apply the previous result to the case of p-relations, we need to extend the 
model of computation. As in the case of Turing machines, the more natural way of doing so is to 
add to the standard auxiliary pushdown automaton an additional two-way read-only input tape 
for the domain part of the relation, leaving the usual one-way input tape for the codomain part 
(Figure |Z2ll. 

The semantic of the computation of the extended one-way auxiliary pushdown automaton 
on input (a, P) can be defined as follows: initially the domain input tape contains a = a\ . . Mn, 
the codomain input tape contains ^ = b\ . . .bm, the pushdown store contains the initial symbol, 
all auxiliary tape cells are blank and all tape heads scan the first (leftmost) cell of the tapes. 

■'without such restriction, as one can verify, such devices become in fact (computationally) equivalent to standard 
Turing machines. 
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Figure 2.4: An extended auxiliary pushdown automaton for recognizing relations. 



The input (a, (3) is said to be accepted iff if the extended 1-AuxPDA, started in the initial state, 
makes a sequence of moves which eventually enters the accepting state. The relation accepted 
by an extended 1-AuxPDA is the set of ordered pairs of strings of input symbols so accepted. 
The semantic for the nondeterministic version can be similarly defined. 

The resources of time, space and ambiguity for the extended model are defined as before, 
but only the codomain input tape is restricted to be one-way (while the domain input tape is an 
unrestricted two-way tape). 



We are ready to give oii r second result on "efficiently" rankable relations. By adapting a 
proof of iBertoni et al.l (11994) . we can prove the following 



Theorem 2.3.1 If a p-relation R is accepted by a polynomial time extended one-way unambigu- 
ous auxiliary pushdown automaton, then r^ G FR 



Proof . Given an extended 1-UAuxPDA A, with state set Q, and some a € £*, it is always 
possible to build (in time polynomial in |a|) a 1-UAuxPDA A'"), with state set 2 x {1, . . . , |a|} x 
£, such that the computation of A^") on input P efficiently simulates the computation of A on 
input (a, p) for every per*. This is made possible by using the state {q,i,ai) of A^"^ to represent 
the automaton A being in state q having the the two-way head on position / reading symbol a, 
(where a = ai . . .a„ for some n € N). It is then clear that if L'") is the language accepted by 
A(«),r^,„,(P)=r«(a,P). 

As it is shown in (' Huynh . 1990 . Theorem 1.1), for every language L accepted by a poly- 
nomial time 1-UAuxPDA, ^^(P) can be computed in IPI'^''^ time for every P € £* by means 
of a sequential algorithm workin g on a suitably defined path system depending on the surface 
configurations of the automaton (ICookl. In order to use this result on A^"' we need a more 

accurate analysis of the constants hidden in the asymptotics. 

As one can verify, the (sequential) algorithm of ( Huynh , 1990l Theorem 1.1), applied on a 
1-UAuxPDA A', works on input p € £* in kN'^qd^l) time for some constant k and a polynomial 
q, depending on the running time of A', where A'^ is the number of nodes of the path system 
built by the algorithm. As an immediate consequence of the particular construction of the path 
system, N = 5'^<7(|3|), where 5' is the number of surface configurations of A'. Recall that a 
surface configuration (ICookL 1 19701) of a 1-UAuxPDA is a tuple {z,a,i,yq5) where: z is the 
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topmost symbol of the stack, a is the symbol on the input tape under the head that is in position 
/, 76 is the content of the working tape and q is the status of the automaton. Thus, 5 = 
where his a constant depending on the cardinality of the alphabets (which here are assumed to 
be constant) and Q' is the set of states of A'. 

We are then able to conclude that, since A^") has 2|2|log|a| states and /? is a p-relation, 
r^(a)(P) can be computed in time ^(2/j|2|p(|a|)log |a|)^^^(p(|a|)) = |a|'^(^l I 

Also in this case, by immediate application of Theorem l2.1.1[ we can introduce, via extended 
one-way auxiliary pushdown automata, a new class of p-relations admitting a polynomial time 
uniform random generation. 

Corollary 2.3.1 If a p-relation /? C £* x £* is accepted by a polynomial time extended one-way 
unambiguous auxiliary pushdown automaton, then it admits a polynomial time uniform random 
generator 



Chapter 3 



Ambiguous Descriptions 



Vous vous etonnez comme cette matiere, brouillee pele-mele, au gre du ha- 
sard, peut avoir constitue un homme, vu qui'il avait tant de choses necessaires d 
la construction de son etre, mais vous ne savez pas que cent millons de fois cette 
matiere, s'acheminant au dessein d'un homme, s'est arretee a former tantot une 
pierre, tantot du plomb, tantot du corail, tantot une fleur, tantot une comete, pur le 
trop ou le trop peu de certains figures qu'il fallait ou ne fallait pas d designer un 
homme? 

Cyrano de Bergerac, Voyage dans la lune 



In this chapter we discuss a new approach to uniform random generation based on the notion 
of ambiguous description. For the sake of simplicity, here we restrict our attention to the case 
of combinatorial structures instead of /^-relations. After a brief restatement of the notions of 
uniform random generator and approximation scheme for this case, we give a formal definition 
of a description of a combinatorial structure, together with its ambiguity function. Then we give 
some general results relating combinatorial structures admitting (possibly ambiguous) descrip- 
tion with uniform random generation, approximate and exact counting. Finally, we discuss two 
very simple applications of this paradigm. 



3.1 Combinatorial Structures 



We start this section by recalling some notions widely used in literature (iFlajoleA Il988h . A 
combinatorial structure is a pair (5, | ■ |), where the domain S is a finite or denumerable set and 
the size | • [ : 5 — N is a function such that #{s £ S : \s\ = n} is finite for every « G N. Here, 
we implicitly assume the elements s £ S admit a (recursive) binary representation such that each 
1^1 is polynomially related to the length of its binary representation; this allows our model of 
computation to manipulate the elements of combinatorial structures. The census fiinction €$ of a 
combinatorial structure (5, | • |) is the function Cj : N — > N such that Cs{n) = #{s £S:\s\=n] for 
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every n G N. In the following, for the sake of brevity, we denote by S„ the subset of the domain 
of a combinatorial structure (5, [ • [) defined as = {s ^ S : \s\ = n} C S (so that Cs{n) = #5„). 

Different kinds of combinatorial structures are usually studied in literature, for instance: 
families of graphs, or trees, where the size can be considered as the number of vertices, or 
edges; formal languages, where the leng th can be taken as th e size of words; discrete geomet- 
rical objects, such as polyominoes (Dele st and Viennotl 11984'), or tilings, where the size can be 



considered as the number of faces, or elementary objects constituting the whole figure. 



By adapting to our case analogous definitions of iJerrum et al. (Il986h recalled in Section [T31 



we now introduce the concepts of uniform random generator and randomized exact counter for 
a combinatorial structure (5, | • |) together with the notion of randomized approximation scheme 
for the census function €$■ From now onwards we assume _L to be a distinguished element not 
belonging to the domain of any combinatorial structure. 

3.1.1 Uniform random generator 

Definition 3.1.1 An algorithm A is a uniform random generator (u.rg.) for a combinatorial 
structure {S,\-\) iff, for every n> such that Cs{n) > 0, 

(i) A on input n gives output A{n) G S„ U {J-}, 

(ii) Pr{A(?i) = s \ A(«) / ±} = \/Cs{n),for every s G S„, and 
(Hi) Pr{A(?i) = ±} < 1/4. 

Observe that the constant 1/4 in the previous definition can be replaced by any positive number 
strictly less than 1 leaving the definition substantially unchanged in the sense of the following 

Lemma 3.1.1 Given a combinatorial structure (5, | • |), let A be an algorithm for which (P 
and d?!]) of Definition \3.1.1\ hold and such that Pr{A(«) = _L} < 5, for some < 5 < 1. Then, for 
every < 6' < 1 there exists an algorithm fi^ for which (0) and ^ of Definition \3.1.l\ hold and 
such that Pr{A'(«) = _L} < 5'. Moreover, if A works in T/^{n) time and uses Rh{n) random bits, 
then A' works in 0{T;^{n)) time and uses 0{R/^{n)) random bits. 

Proof . If 5 < 5' the statement is trivial. Otherwise, let A' be algorithm 13. II 



input n 

5' ^ ±, / ^ 

while / < [log 6'/ log 8] and ^ = _L do 

/ / + 1 

s ^ A{n) 
output s. 



Algorithm 3.1: An algorithm to increase the success probability of a u.r.g. 
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To prove the correctnes^ of A', let n be such that Cs{n) > 0, then 

Pr{A'(?i) = ±} = Pr{A(«) = ±}ri°g8'/iog81 < gRogS'/iogSl ^ g/. 

on the other hand, as it is easy to verify, Pr{A'(?i) = s \ A'{n) 7^ _L} = Pr{A(?i) = s \ A{n) / ±} 
for every ^ £ S„. Finally, the statements about the computation time and number of random bits 
used by A' follow immediately from its definition. I 



3.1.2 Randomized approximation sclieme 



Following IStockmeyerl (119831 ): iKarp et al.l (Il989r) . we now introduce the notion of approximate 
counting for combinatorial structures. 

Definition 3.1.2 An algorithm A is a randomized approximation scheme (r.a.s.) for the census 
function Cs of a combinatorial structure (S, | • |) ijf, for every n > such that Cs{n) > and 
every £ G (0, 1), 

( i) A on input n,e gives output A{n, s) € Q U {_L}, 

(ii) Pr{(l -£)Q(«) < A(«,£) < (1 +£)C5(«) | A(?i,£) / -L} > 3/4 and 
(Hi) Pr{A(?i,£) = -L} < 1/4. 



Moreover, ar.a.s. is said to be a fully polynomial time r.a.s. whenever it works in time polynomial 
in n and 1 /£. 



Observe that even in this case the constant 1/4 of point f ITnl) can be replaced by any positive 
number strictl y less than 1 by esse ntially the same argument of Lemma [3.1.1[ moreover, follow- 
ing an idea of Jerrum et al. ( 19861) . the constant 3/4 of point dn]) can be replaced by any number 
strictly between 1 /2 and 1 again leaving the definition substantially unchanged in the sense of 
the following 



Lemma 3.1.2 Given a combinatorial structure (S, | • |), let A be an algorithm for which (13) 
and f ITnl) of Definition \3.1.2\ hold and such that Pr{(l —z)Cs{n) < A{n,e) < (1 + e)Cs{n) \ 
A(«,£) 7^ _L} > 1/2 + 6, for some < 5 < 1/2. Then, for every < 6' < 1, there exists an 
algorithm A' for which (0) and f lnTl) of Definition I J. 7.2] hold and such that Pr{(l —e)Cs{n) < 
A'(n,e) < (1 +e)Cs{n) | A'(?i,£) 7^ -L} > 1—5'. Moreover, if A works in T/i^(n,e) time and uses 
/?a(«)£) random bits, then A' works in 0{T/i,{n,e)) time and uses 0{Rfi,{n,e)) random bits. 

Proof . If 5' > 1/2 — 5 the statement is trivial. Otherwise, let A' be Algorithm [3]2j where K > 
is an integer constant whose value will be determined in the following. 

The statements about computation time and number of random bits of A' follow immediately 
from its definition; we prove in detail only the correctnes^ of A'. Fix n such that Cs{n) > and 
let / be the random variable representing the value of j at the end of the execution. It is easy to 
observe that 

Pr{A'(?i,£) = ±}= Pr{7 = 0} = Pr{A(?i,£) = ^ji^rs-^iogS'-'l ^ 

'for a detailed discussion of the probabilistic aspects of such proof, see Appendix lA.2.21 
^for a detailed discussion of the probabilistic aspects of such proof, see ApDendix lA.2.21 



32 



Ambiguous Descriptions 



§3.1 



input n,e 

while / < K[6"^log5'"^] do 
/ ^ / + 1 

c A(?l,£) 

if c / ± then 

Cj c 

if j>Othen 

output the median of ci,...,Cj 

else 

output ±. 



Algorithm 3.2: An algorithm to increase the probability of correct approximation of a r.a.s.. 

Consider now the case 7 > 0. Let C\,.. . ,Cj be the random variables representing the values 
assumed by ci , . . . , at the end of the execution and let M be the random variable corresponding 
to the median of Ci, . . . ,Cj. It is then evident that if M ^I{n,e) = [(1 —e)Cs{n),{l +e)Cs{n)], 
then at most one half of the C,'s with 1 < / < 7 are such that C, € I{n,e). Hence, by Lemma lA.2.11 
and for every k > 0, Pr{M ^ I{n,e) \ J = k} < e^'^^^' where Pr{C,- G /(«,e)} = Pr{A(«,£) G 
/(«,£) 1 A(?i,£) /_L} > 1 /2 + 6 by the definition of A. Hence, if = Kfa^^logS'"'], 



where q = Pr{A(?i, s) 7^ _L} > 3/4 and the last inequality follows from the fact that \ —e ^ >x/2 
for X G [0, 1]. Finally, by Lemma lA. 1.1[ the integer K > can be chosen such that 



3.1.3 Randomized exact counter 

Definition 3.1.3 An algorithm A is a randomized exact counter (r.e.c.)for a combinatorial struc- 
ture (5, [ • j) ijf,for every n> such that Cs{n) > 0, 



Pr{A'(?i,£) /(«,£) I A'(w,£ 



Pr{M0 /(«,£) n7>0} 

Pr{7 > 0} 
Lf^i Pr{M ^I{n,e)\J = k} Pr{7 = k} 
Pr{7 > 0} 
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(i) A on input n gives output A{n) G N U {-L}, 

(ii) Pr{A(?i) = Cs{n) \ A{n) ^ ±} > 3/4 and 
(Hi) Pr{A{n) = ±} < 1/4. 

Also for this definition, one can show that the choice of the constants 1/4 and 3/4 is not restric- 
tive by reasoning as in Lemnia [3.1.1l and r3.1.2l 

3.1.4 An example: regular languages 

We now give a very simple example of the definitions given in this section. Let us consider a 
deterministic finite automaton si = {Q,qo,5,F) over a finite alphabet £, where Q is the set of 
states, qo ^ Qis th e initial s tate, 5 : Q x £ — > Q is the transition function and F C g is the family 
oi final states (Hop croft an d Ullmanl 1 1 9791 '). Let Lj^ C £* be the language recognized by £/. 



Hence, the combinatorial structure we consider is {L^ ,\-\), where | • | denotes the length. Our 
aim is to design a u.r.g. for such a structure. 

The uniform random generation algorithm first computes, for every q ^ Q and \ < I <n, 
the number Cq{t) of all words of length I accepted by {Q,q,b,F). This computation takes 
0{n^) time on a RAM under logarithmic cost criterion since each |C^(£))/>i is the sequence of 



coefficients of a rational generating function (IChomsky and Schiitzenbergen . Il963h . For every 



of these terms, the algorithm also computes the number bq{£) = [logC^(^)] of bits required to 
represent {I, . . . ,Cq{£)} which can be obtained in C?(^log^) time (see Lemma P^. 1.31 ). Hence, 
such a precomputation phase requires 0{n^logn) time. 



Then, the algorithm executes the procedure Generate(go5'^) described by Algorithm | 
Here, k G N is a global parameter whose value will be determined in the following, while £ and 
all the sets {{s,o) ^ Q xL: 5{q,o) = s}, for g € 2, are endowed with some total order relation 

Reasoning by induction on £, it is easy to verify that if Generate(g,^) returns an output w 
different from _L, then w is uniformly distributed in the set of words of length i accepted by 
{Q,q,?>,F). An upper bound to the probability that Generate(^,£) gives output _L depends on 
the global parameter K. Denoting by e{i) the maximum of all probabilities that Generate (^,^) 
gives output _L for ^ € Q, one can easily show that e(l) < (1/2)"^ and e{£) < (1/2)*^ + e{£ — 1) 
for every £ > I. A simple induction proves e{n) <n/2^ and hence, for every constant f > 0, by 
fixing K = t+ [logn] we obtain e{n) < 1 /2'. 

As far as the time complexity is concerned, if we denote by T{£) the maximum time cost 
of procedure Generate(g,£) for q ^ Q, for every I < i <n, one can write the recursion T{i) = 
0{k-£) + T{£-1) which proves that T{n) = Oiyin^) = 0{{t + \ogn)n^). 

We summarize our discussion by the following 

Proposition 3.1.1 Given a deterministic finite automaton s/, there exists a PrRAM which, on 
input « E N, generates a word uniformly at random in PI with probability 1 — 1 /2' working 
in 0{n^{t + \ogn)) time. Hence the combinatorial structure (L^, | • |) admits a u.rg. working in 
0{n^\ogn) time. 



3.2 Ambiguous Description 



We formally introduce the concept of (ambiguous) description (see Figure [3TT]) : 
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procedure Generate (^,^) 
/ ^ 0, r ^ _L, w ^ _L 
while / < K and r=±do 
/ ^ / + 1 

generate m € {1, ... ,2^'' (^'} uniformly at random 
if M < Cq{l) tlien r ^ M 
if r ^ ± then 

if n = 1 then 

let a be the r-th symbol in the set {a G £ : h{q,o) € F} 
w -i— a 

else 

choose the smallest element {p,a) in the set {(5,ct) € 2 x £ : h{q,o) = s] 
such that C,(£-l)>r 

(s,a)^{p,a) 

Wp -f— Generate — 1) 
if Wp 7^ _L then w ^ awp 

return w. 



Algorithm 3.3: a uniform random generator for regular languages. 

Definition 3.2.1 {T ,\-\) is a description o/(S, [ • j) via the function f :T ^ S, iff is a surjective 
function preserving \ ■ \, i.e., \f{t) \ = \t\for every t ^T. The ambiguity of the description is the 
function d : S —^N defined by d{s) = #{t G T : f{t) = s}, for every s & S. We say that the 
description is ambiguous if d[s) > 1 for some s £ S. Moreover, the description is said to be 
polynomial whenever f and d are computable in polynomial time and there exists some Z) € N 
such that d{s) = 0{\s\^). 




Figure 3.1: An Example of Ambiguous description. 



In this thesis, we will give examples of (ambiguous) descriptions, such as the family of 
derivation trees of a grammar as a (possibly ambiguous) description of the strings derived in 
the grammar (see Section |4TT]) and the family of computations of a nondeterministic device as a 
(possibly ambiguous) description of the strings accepted by the device (see Section l42l ). 
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3.2.1 Uniform random generation 



We start by considering the uniform random generation problem. 



Theorem 3.2.1 If a combinatorial structure {S,\-\) admits a polynomial (ambiguous) descrip- 
tion {T, I • I) and there exists a polynomial time u.rg. for {T, | • |), then there exists a polynomial 
time u.rg. for {S, \ ■ |). 



To pro ve this theorem, we show a stronger result derived by applying the lKarp. Luby. and Madras 
technique for sampling from a union of sets. 



Lemma 3.2.1 Let {S, \ ■ |) admit a polynomial (ambiguous) description {T , \ ■ |) via the function 
f and let Tf{n) and Td{n) be respectively the computation time of f and of its ambiguity d; 
moreover, assume d{s) = 0{\s\^) for some D G N. If B is a u.rg. for (T, | • |) working in TQ{n) 
time and using Rsin) random bits, then there exists a u.rg. for (S, | • |) working in 0(n^^ + 
n^iTsin) + Tf{n) + T(i{n))) time and using 0{n^^ -\-n^RQ{n)) random bits. 

Proof . Define, for the sake of brevity, D{n) = Ki?i^ + K2 where Ki , K2 are two integer constants 
such that (jf(i') < Dd^l); let A be Algorithm [33J where K > is an integer constant whose value 
will be determined in the following. 



input n 

m lcm{l, . . . ,D(?i)}, ^ [logm] 
/ ^ 0, 5 ^ _L 

while / < KD{n) and 5 = ± do 
/ ^ / + 1 
t ^ B{n) 
iff /_L then 

generate r G {1, ... ,2^} uniformly at random 
if r > m/d{s) then 
s^± 

output s 



Algorithm 3.4: An u.r.g. for (ambiguously) described combinatorial structures. 

First of all, we focus on the computation time of A on input n; the time to precompute 
D{n), m and £, by Lemma lA.l. 21 and lA. 1.31 equals 0{n^^) and the time of each iteration equals 
0{TQ{n) + Tf{n) + Td{n)) plus the time 0{n'^) required for the generation of r. Moreover, each 
iteration uses at most Rsin) and 0{n^) random bits to compute t and r respectively, hence the 
total amount of random bits used by A adds up to 0{n^^ + n^Rein)). 

Now, we prove the correctness of A. Assume that Cs{n) > so that, since / is surjective, 
Cr(n) > and let S and T be the random variables representing respectively the value of s 
and t at the end of a while iteration (observe that S and T are well defined since their value is 
independent of the outcomes of the preceding iterations). Then, S takes value s € S„ whenever 
T € (s) C r„ and r < m/d{f{T)). Moreover, by definition of B, there exists some < 5 < 
1/4 such that Pr{r = = (1 — 5)Cr(?i)^' for every t G r„ and, as it is easy to verify since r 
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is independent of S and T, Pr{r < m/d{f{T)) \ T = t} = d{f{t)y'^m2-^^"^'"'^ . Hence, for every 

Pr{S = s}= £ PT{T = t,r<m/dif{T))} 

tef-'{,s) 

= £ Fr{r<m/d{f{T))\T = t}Pr{T=t} 

fe/-'(-v) 

= d{s) (t/(^)-'m2-ri°gH (1 _ 6)Cr (n)-') 
= (l-5)m2-ri°gHcr(„)-i 

wliich is independent of s. On the otiier hand, at the end of each while iteration, Prj^ = ±} = 1 — 
Pr{S eSn} = l-Cs{n){{l- 5)m2-ri''g'"lCr(«)"^). Then, since m2-ri°gH > 1/2 and Cr(«) < 
Cs{n)D{n), 

Fr{S = ±}<l-l-^. 
Finally, since Pr{A(«) = ±} = Pt{S = _L}'^("), the integer K > can be chosen such that 

3 y \ kD{«) 



and hence Pr{A(?i) = ±} < 1 /4. 



Moreover, as it is easy to verify, for every s G 5„, it holds Pr{A(«) = s \ A{n) = 1.} = Pr{S = 
5 I 5 7^ _L} and, since Pr{S = s} is constant, it is immediate to conclude that Pr{A(«) = s \ A{n) ^ 
±}=C5(«)-1.I 

In particular, in the case of finite ambiguity, it holds that 

Corollary 3.2.1 Under the same hypotheses as in Lemma I J. 2.71 with the stronger condition 
d{s) < D' for every s G S and some D' € N, there exists a u.rg. for (S, j • |) working in time 
0{TQ{n) + Tf{n) + Tii{n)) and using 0{RB{n)) random bits. 



3.2.2 Randomized counting 

Now, let us consider the approximate and exact counting problem. 

Theorem 3.2.2 Let (S, | • |) be a combinatorial structure admitting a polynomial (ambiguous) 
description {T , \ ■ |) and assume there exists a polynomial time u.rg. for (T, j • j). If Cj{n) is 
computable in polynomial time, then there exists a fully polynomial time ra.s.forCs. If moreover 
Crin) is polynomially bounded, then there exists a polynomial time re. c. for (5, | • |). 

To prove thi s theo rem, we show two stronger results essentially obtained by an application of 



Hoeffding's (1 19631) inequality. 



Lemma 3.2.2 Under the same hypotheses as in Lemma \3.2.1\ if Cj is computable in Tcj (n) 
time, then there exists a r.a.s. for €$ working in 0{Tcj{n) +n^^z^^{TQ{n) + Tf{n) + Tj{n))^ 
time and using 0{n^^z^^RQ{n)) random bits. 
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input n 

i^O, j ^0,m^0 
while / < K\D{n)/eY do 

/ ^ / + 1 

t ^ B{n) 

iff /±then 

m^m + \/d{f{t)) 

if j > then 

output mCrin)/ j 

else 

output ±. 



Algorittim 3.5: A r.a.s. for ambiguously described combinatorial structures. 

Proof . Define, for the sake of brevity, D{n) = Kin^ + K2 where Ki , K2 are two integer constants 
such that d{s) < D{\s\); let Abe, Algorithm 13. 51 where K > is an integer constant whose value 
will be determined in the following. 

The statements about computation time and number of random bits of A follow immediately 
from its definition; we prove in detail only the correctnes^ of A. Fix n such that Cs{n) > and 
let J be the random variable representing the value of j at the end of the execution. It is easy to 
observe that, 

Pr{A(«) = ±} = Pr{7 = 0} = Pr{B(n) = A_}^Wn)/^r- ^ 
Consider now the case / > and define for the sake of brevity 

^, = E{l/d{mn)))\B{n)^±)= ^ l/d{f{t))CT{n)-' 

teT„ 

= Cs{n)CT{n)-\ 

If M is the random variable representing the values assumed by m at the end of the execution 
and I{n,e) = [(1 - £)fi, (1 + £)fi], by Lemma|ATl Pr{M/J 1{n,e) \ J = k} < 2e"2<:(A'E)\ for 

■'for a detailed discussion of tlie probabilistic aspects of such proof, see Appendix lA.2.21 
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every A: > 0. Hence, ifN = K \D{n) /s] ^, 



Pr{M//0 /(«,£) |/>0} 



Pr{M/7^ /(?!,£) n/>0} 
Pr{7 > 0} 

If^j Pr{M/J ^I{n,£)\J = k} Pr{7 = k} 
Pr{J > 0} 




where q = Pr{B(«) 7^ ±} > 3/4 and the last inequahty follows from the fact that I —e ^ > x/2 
for X G [0, 1] and that Cr(?i) < Cs{n)D{n). 

Finally, by Lemma lA.l. li the integer K > can be chosen such that 



Then, since A{n,e) = MCr(«) /J, it holds 

Pr{M/7G /(?!,£) |7>0} = Pr{(l- £)Cs(«) <A(n,£) < (1+£)C5(«) | A(«,£) ^1}, 
hence, Pr{(l-£)C5(«) <A(?i,£) < (l+£)C5(n) 1 A(?i,£) / ±} > 3/4. I 
We now consider the problem of obtaining a r.e.c. 

Lemma 3.2.3 Under the same hypotheses as in Lemma \3.2.2\ there exists a re.c. for {S, \ ■ |) 
working in 0{Tcr{n) +n^^CT{nf{TB{n) + Tf{n) + Td{n))) time and using 0{n^^CT{nYRB{n)) 
random bits. 

Proof . Using the same notation of Lemma [3.2.2[ define the algorithm A' as 



Since / is surjective Cr(?i) > Cs{n), hence A' is correct since \IK{n,t{n)) /Cs{n) — 1 1 < 1/ (3Cr [n)) 
implies |A(?i,£(?i)) —Cs{n)\ < 1/3. Finally, the statement about computation time and number 
of random bits used follows immediately from Lemma 13.2.21 I 
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In particular, in the case of finite ambiguity, it holds that 

Corollary 3.2.2 Under the same hypotheses as in Lemma \3. 2. 21 with the stronger condition 
d{s) < D' for every s £ S and some D' G N, there exists a ra.s. for Cs working in 0(Tct ('^) + 
£~^(7b('^) + Tf{n) + Td{n))^ time and using 0(£^^/?b(«)) random bits and a r.e.c. for {S, \ ■ |) 
working in 0{Tc-^{n)+CT{nY{TQ{n) + Tf{n) + Tii{n)Y) time and using 0{CT{nYRQ{n)) random 
bits. 

3.3 Simple Applications 

In the remaining part of this chapter, we give two simple applications of the above results to the 
case of formal languages over a given alphabet £, i.e., for combinatorial structures (5, | • |) such 
that S C E* and | • | is simply the word length. Given two such combinatorial structures (S, | • |) 
and {T, \ ■ |), it is natural to define the union as {SUT,\ - 1) and the product as (S- T, | • |). 

We start from the uniform random generation of the product: 

Theorem 3.3.1 Let {S,\-\) and {T,\-\) be two combinatorial structures such that 5 C £* and 
r C £* are recognizable in polynomial time. If both structures admit a polynomial time u.rg., 
then there exists a polynomial time u.rg. for (5 • T, | • |). 

Proof . If P = S xT denotes the Cartesian product of S and T, then {P,\-\) is a polynomial 
(possibly ambiguous) description of (5 • T, | • |) via the function f{{s, t)) = s-t, for every {s, t) € 
P; the ambiguity is defined as 

d{q)= E Xs(^)Xr(0 

seS,teT:s-t=q 

for every q^S-T , where Xa denotes the characteristic function of the set A. Then, if A and B are 
the polynomial time u.r.g. respectively of (S, | • |) and (T, | • |), the integer K > can be chosen 
such that Algorithm [3]6] is a polynomial time u.r.g. for (P, | • |). 



input n 

i^O,p^l. 

while / < K and p = ±do 

i^i+l 

s ^ A{n), t ^ B{n) 

if 5 / _L and t ^ ± then 

p^{s,t) 

output p. 



Algorithm 3.6: A u.r.g. for the product of combinatorial structures. 

Therefore, by Theorem l3.2.1[ there exists a polynomial time u.r.g. for (5 • T, | • |). I 

Now we consider the case of the union: 

Theorem 3.3.2 Under the same hypotheses as in Theorem \3. 3.1\ ifCs and Cj are computable 
in polynomial time, then there exist a polynomial time u.r.g. for (5U T, | • [). 
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Proof . If U = S ®T denotes the disjoint union of S and T, then {U, | ■ |) is a polynomial 
(possibly ambiguous) description of (SU T, | • [) via the identity function (we denote here by 
/) and the ambiguity is J = Xs + Xr- As before, the integer K > can be chosen such that 
Algorithm 13.71 is a polynomial time u.r.g. for (f/, [ • [), where A and B are the polynomial time 
u.r.g. respectively of (S, | • |) and (r, | • |). 



input n 

i^O,i^ riogCs(«)+Cr(«)l,M^-L 
while / < K and m = _L do 
/ ^ / + 1 

generate r G { 1 , . . . , 2^} uniformly at random 
\^r <Cs{n) then 
u ^ IK[n) 
else ifr <Cs{n)+CT{n) then 

U ^ B(?2) 

output u. 



Algorithm 3.7: A u.r.g. for the union of combinatorial structures. 

Again, by Theorem 13.2. 1[ there exists a polynomial time u.r.g. for (SU T, | • |). I 

Finally, since Cp{n) = Cs{n)CT{n) and Cu{n) = Cs{n) + CT{n), from Theorem l3.2.2l simply 
follows 

Theorem 3.3.3 Under the same hypotheses as in Theorem \3.3.2\ there exist and a fully polyno- 
mial time r.a.s. for the census functions CsuT ond Cj s- 



Chapter 4 
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Formal Languages 



A questo piacere contribuisce la varieta, Vincertezza, U non veder tutto, il po- 
tersi percid spaziare coU'immaginazione, riguardo a aid che non si vede. . . E piace- 
volissima ancora .. .la vista di una moltitudine innumerabile, come delle stelle, o di 
persone ec. un moto molteplice, incerto, confuso, irregolare, disordinato, un ondeg- 
giamento vago ec, che Vanimo non possa determinare, ne concepire definitamente 
e distintamente . . . 

Giacomo Leopardi, Zibaldone di pensieri 



In this chapter we present some nontrivial appUcation of the general paradigm introduced in 
the previous chapter to some class of formal languages. In particular, we show how to obtain, 
under suitable hypotheses, a polynomial time uniform random generator and a fully polynomial 
time randomized approximation schemes for polynomially ambiguous context-free languages, 
languages accepted in polynomial time by nondeterministic one-way auxiliary pushdown au- 
tomata of polynomial ambiguity and, finally, polynomially ambiguous rational trace languages. 



4.1 Context-Free Languages 



A natural example of our general paradigm is given by the family of derivation trees of a 
context-free grammar, considered as (possibly ambiguous) description of the corresponding 
language. Thus, we can design simple u.r.g. and r.a.s. of census functions for inherently am- 
biguous context-free languages. These procedures work in polynomial time whenever the de- 
gree of ambiguity of the associated grammar is polynomially bounded. Moreover, if such a 
degree is bounded by a constant, both procedures require 0{n^\ogn) time on input n under 
logarithmic cost criterion, the same order of growth required by the best known algorithms 



(IFlajolet. Zimmerman, and Van CutsemLll994l : lGoldwurm 



1995h for the analogous problems in 



the unambiguous case (once their time cost is adapted to the PrRAM model of computation). 
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To show this application in detail, consider a context-free grammar (c.f.) G = {¥,11,5,?), 
where V is the set of nonterminal symbols (also called variables), £ is the alphabet of terminals, 
S € V is the initial variable and P i s the family of productions. We assume G in Chomsky normal 



form (iHopcroft and UllmanLll979|) without useless variables, i.e., every nonterminal appears in a 
derivation of some terminal worcu Moreover, for every A € V, let be the family of derivation 
trees with root labelled by A deriving a word in £+. It is easy to see that there are finitely many 
t ^ Ts deriving a given x € in the following, we denote by ddx) the number of such trees 
and call ambiguity of G the function • N — N defined by dgin) = maxxez" dcix), for every 
n € N. The grammar G is said finitely ambiguous if there exists a A; € N such that ddn) < k 
for every n > 0; in particular, G is said unambiguous if A: = 1. On the other hand, G is said 
polynomially ambiguous if, for some polynomial p(n), we have dG{n) < p{n) for every n>0. 

Clearly, our idea is to use the structure {Ts, | • |) as a (possibly ambiguous) description of 
the language L generated by G, where, for every t G Ts, \t\ is the length of the derived word. 
However, to get a u.r.g. for L and a r.a.s. for its census function, we first need two preliminary 
procedures: one for generating a tree of given size in Ts uniformly at random, the other for 
computing the degree of ambiguity dcix) for the words x G £+. 



4.1.1 Uniform random generation of derivation trees 



A u.r.g. for derivation trees can be designed by followin g a general approach to the uniform 
random generation of combinatorial structure s proposed by iFlaiolet et all (119941) . The algorithm 
we obtain is similar to the procedure given by GoldwurmI ( 1995 ) for generating words uniformly 
at random in unambiguous c.f. languages. Here, we essentially adapt that routine to our case 
and evaluate its time complexity with respect to our model of computation. 

To fix the notation, for every A £V and £ E N, let 7^ be the subset {t ^ Ta : \t\ = £}, and 
let Ca{£) be its card inality. It is well known that every s equence {Ca(^)}^>i has an algebraic 
generating funct ion (IChomsky and Schiitzenbergeil . Il963h and hence, applying Comtet's recur- 
rence equation ( Comtet , 1 1964 ). its first n terms Ca(1), • • • ,CA(n) can be computed in 0{n^) 
time on a RAM under logarithmic cost criterion. For each Ca(^) with I < £ <n, the integers 
bA{£) = [logCA(^)] are then computed, by Lemma lA. 1.31 in O(^log^) time, for an overall time 
cost of 0{n^logn). Observe that this precomputations has to be executed only once even if we 
have to generate several derivation trees in T^. 

Then, to generate a tree uniformly at random in 7J, we apply Algorithm 14. 1 1 which works on 
input {A,£) G y X N and uses a global parameter K (depending on n but not on £), whose value 
will be determined in the following. In the procedure. Pa denotes the subset of productions in P 
of the form A— >SC with B,C GV, and P\ denotes the set of productions in P of the form A— 
with a gL. 

The procedure chooses an element uniformly at random either in P\ (if ^ = 1), or in the set 
Pa X {I, ■■■,£— 1} (ii £ > I), the choice depending on a total order relation < among the elements 
of these sets. To define <, we assume a lexicographic order =^lex in both alphabets £ and V and 
set A^a < A-^b in if a ^lex b, while {A^BC,h) < {A^DE,k) in Pa x {1, 1} if 

either h < k, ox h = k and BC ^lex DE- 

Reasoning by induction on £, it is easy to verify that, if the output A (^) of RandomTree(A,^) 
is different from _L, then A{£) is uniformly distributed in T^. More precisely, for every t G T^, 



'it is well known that every c.f. language not containing the empty word e can be generated by such a grammar. 
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procedure RandomTree(A,£) 
j 0, r _L, ? _L 
whUe / < K and r = _L do 

generate u £ {I, . . . ,2^^^^^ uniformly at random 
if M < Ca(^) then r •<— m 
if r 7^ ± then 

if^ = 1 then 

let A^a be the r-th element of P\ 
t <(- {A, a) 

else 

compute the smallest element {A^BC,k) inPAxjl,...,^— 1} 

such that CD{h)CE{£-h)>r {*) 

{A^DE,h)<{A^BC,k) 

tB •<— RandomTree(B,^) 

tc ^RandomTree(C, i — k) 

if 7^ ± and /c 7^ -L then t ^ {A,tB,tc) 

return t. 



Algorithm 4.1: a uniform random generator of derivation trees. 

we have 

Pr{A(n) =t\A{n) 7^ ±} = I/CaW- 

Now, let us give an upper bound to Pr{A(?i) = -L}. Clearly, this value depends on the global 
parameter K used by the procedure. Denoting by e{i) the maximum of all values Pr{A(^) = _L} 

for A eV, one can easily show that e{l) < (1/2)^ and e{i) < {\ /2)^ + maxi<k<(:-i{e{k) + 
e{£ — k)} for every I <£ <n. A simple induction proves e{n) < (2n — l)/2'^ and hence fixing 
K = 3 + [logn] we have 



Pr {A(n) = ±} < e{n) < 1 /4 for every AeV. 
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As far as the time complexity is con cerned, we assume t o search the element {A^BC,k) of 
step (*) by a boustrophedonicH routine (^ Flajolet et al.L 1994). Again, let A'^(^) be the maximum 
number of PrRAM instructions executed by RandomTree(A,^) for A G V. Then, for a suitable 
constant c > 0, one can write the following recursion, for every I <£ <n, 



0(K) 

6>(k) + maxi<y<£{cmin{7, 



if ^ = 1 

-j}+N{j)+N{£-j)} ife>l. 



This proves that N{n) = 0{Kn) + cf{n), f{n) being the solution of the minimax equation /(«) = 
max{/(/:) +/(« — /:) + min|fc,?2 — usu ally arising in the evaluation of the cost of boustro- 
phedonic search ( Greene and Knuthl. 1981 '). Since it is known that f{n) = 0{n\ogn), assuming 
K = O(logn), we get N{n) = 0{n\ogn) which, under our model of computation, gives a to- 
tal time cost 0{n^\ogn), since all integers involved in the calls of this routine have 0{n) bits. 
Finally, as it is straightforward to check, the number of random bits used by the procedure is 
0{n^\ogn). 

We observe that, following an idea described by GoldwurmI ( 1995 ). the computation of the 
coefficients {(Ca(^),^a(^)) : A G V, 1 < £ < «} and the actual process of generating a tree uni- 
formly at random in can be mixed together into a unique procedure which only requires 
space 0{n) under logarithmic cost criterion (leaving unchanged the order of growth of the time 
complexity). 

We summarize the result of this section by the following 



Proposition 4.1.1 Given a context-free grammar G = {V,'L,S,P) in Chomsky normal form, 
there exists a u.rg.for {Ts, | • |) working in 0{n^logn) time and using 0{n^logn) random bits. 



4.1.2 Farley's algorithm for counting derivations 



The number of derivati on trees of a te rminal string in a c.f. grammar can be computed by adapt- 
ing Farley's algorithm (iFarley . for context-free reco gnition. The mai n advantage of this 
procedure, with respect to the well known CYK algorithm ( Harrison . 19781) . is that in the case 
of a grammar with boun ded ambiguity , the computation only re quires quadratic time on a RAM 
under unit cost criterion (lFarleylll97Cll : lAho and Ullmanlll972h . 



Let again G = {V,L,S,P) be a c.f. grammar in Chomsky normal form without useless vari- 
ables. In passing, we note that the algorithm can be easily modified to work on every c.f. 
grammar without £ and unit productions. Our algorithm manipulates a weighted version of 
the so called dotted productions of G, i.e., expressions of the form A— >a • P, where A G V, 
a,p G {LUVy andA^ap gP. 

Given an input string x = aia2. ■ - an, the algorithm computes a table of entries Sij, for < 
i ^ j ^ n, each of which is a list of term of the form [A— • P,?], where A^a • P is a dotted 
production in G and t is a positive integer. Fach pair [A— • P,?] is called state and t is the 
weight of the state. 

The table of lists Sij computed by the algorithm has the following properties for every pair 
of indices <i < j <n: 



1. Sjj contains at most one state [A— j-a • P,?] for every dotted production A— >a • P in G; 



i.e., turning like oxen in plougiiing (Webster). 
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2. a state [A— )-a • (3,f] belongs to Sij iff there exists 5 G V* such that S ^ a\ .. .atAb and 
a =4> a,+i . . Mj; 

3. if [A— )-a • belongs to Sij, then ? = #{a =4> a;+i . . •£?;}, i.e., the number of leftmost 
derivations a 4> <3;+i . . . aj. 

Note that, since there are no £-productions, [A^a • P,?] € 5;.; implies a = £ for every <i <n. 
Furthermore, once the lists Sij are completed for every < i < j < n, the number of parse trees 
deriving x can be obtained by the sum T,[s^AB-.t]eSoJ- 

The algorithm first computes the list 5o.o of all the states [A— )■ • a, 1] such that 5 =4> A6 for 
some 8 G V*. Then, it executes, for I < j <n, the cycle of Scanner, Predictor and Completer 
loops given in Algorithm 14.21 computing, at the j-th loop, the Msts Stj for < / < j. To this 
end the procedure maintains a family of sets L5 , for B G V and 1 < / < j; each Lbj contains 
all indices k < i such that a state of the form [A— )-a - BP,?] belongs to Skj for some A G V, 
a, P G y U {s}, f G N. Moreover, during the computation every state in Sij can be unmarked or 
marked according to whether it can still be used to add new states in the table. 

The statement "ADD D TO Sij" simply appends the state D as unmarked to Sjj and updates 
Lbj whenever D is of the form [A^a • BP, ; the statement "UPDATE [A^a • P, f] in Sij" replaces 
the state [A— )•« • P, m] in Sij for some m G N by [A— >a • P, ?] , finally, "MARK D IN Sij" transforms 
the state D in Sij into a marked state. 

We are then able to prove the following 



Lemma 4.1.1 The table of lists Sij, < i < j <n, computed by the procedure described above 
satisfies the properties [7]), |2]) and\3]il. 



Proof . First, observe that statement [B is easily verified: at line [5] distinct states are added to 
an initially empty list Sij; at lines [131 [T8j [22] a state is added to a list provided no state with the 
same dotted production already appears in the list. 

Statement [2]l only refers to dotted productions appearing in the lists and does not concern the 
weight of the states. Moreover, disregarding the computations on the weight of the states, the 
procedure works on the dotted production exactly like F arley 's algorithm; hence statement [21) is a 



consequence of its correctness (for a detailed proof see (lAho and Ullmanll 19721. Theorem 4.9)) 



Now, let us prove statement [3]). First note that all states in Sij, for 1 < / < j < «, are 
marked during the computation. Hence, we can reason by induction on the order of marking 
states. The initial condition is satisfied because all states in each Sjj, < j <n, are of the form 
[A— > • a,t] and have weight t = I. Also the states of the form [A— )-fl;-,f] have weight t = I and 
again statement ^ is satisfied. 

Then, consider a state D = [A— >a • BpjVv] G Skj with k < j. We claim that w is the number 
of leftmost derivations oB =4> ak+y ...aj. A state of this form is first added by the Completer 
at Une [T3j This means that there exists a set of indices 4 such that for every / G 4 there is 
[A— )-a • Bp, Ui] G Sk,i with G N, and a family Ui of sates [B^y-] G Si^k such that 

W = £ £ tUi. 

ieh [B^r,t]eUi 

Observe that k < i < j for every / G 4 and Ui is the subset of all states in Sij with a dotted 
production of the form B^y-. Moreover, each state [A^a-Bp,M,] G Stj is marked at line [16] 
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1 for j = 1 . . . « do 

Scanner. 

2 for / = 7 — 1 ... do 

3 for [A-)- - a,?] G do 

4 MARK [A^ ■ a,t] IN 

5 if a = aj then add [A^a-,t] TO S', ;- 

6 for / = 7 - 1 ... do 

7 for [B^Y-,f] g5,-j do 

8 MARK [B^J-,t] IN ^/j 

9 for ^ G Lbj do 

10 for [A^a • Bp, m] G S'a:^- do 

11 if [A^aB-P,v] g5^j 

12 then UPDATE [A^aB • P, v + Jm] in 5^:^- 

13 else ADD [A-^aB ■ ^,tu] to 5'^:^ 

Predictor. 

14 for / = ... 7 - 1 do 

15 for [A^a • BP, G 5,;^ do 

16 MARK [A^a-BP,f] IN Si J 

17 for B^y G P do 

18 if [B^ • Y, 1] 'S'yj- then add [B^- • y, 1] to 5^^- 

19 while 3 UNMARKED [A-> -BP,?] G Sjj do 

20 MARK [A^ • Bp, ?] IN Sjj 

21 for B^Y G P do 

22 if [B-^ • y, 1] 5^' J- then add [B^ • Y, 1] to 5^,^ 



Algorithm 4.2: Part of Barley's algorithm modified to count derivation trees. 

or l20l before D is added to Skj- Also all states in Ui, for all i £ Ik, are marked during the 
computation of the weight w. Observe that, due to the form of the grammar, updating such a 
weight w cannot modify the weight of any state in Uj. As a consequence all the sates in Ui are 
marked before D. Hence, by inductive hypothesis, we have for every / G 4 

M,- = #{a =4> fljt+i . . .a,} (4.1) 

and, for each [B^y-,t] G Ui, 

t=#{y^ai+i...aj}. (4.2) 
Now the number of leftmost derivations aB ^ a^+i ■■■cij is clearly given by 

52 #{a 4>ayt+i ...a,-} ^ #{y^a;+i ...ay} 

k<i< j B^yeP 

and the claim follows from statement [D and equalities (14.11 ) and (14.21) . I 
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Theorem 4.1.1 Given a context-free grammar G in Chomsky normal form and assuming the 
RAM model under logarithmic cost criterion, the algorithm described above computes the num- 
ber of derivation trees of an input string of length n in 0{n^) time, consuming 0{n^) space. If 
the grammar G is finitely ambiguous, then the algorithm has time complexity 0{n^\ogn) and 
space complexity 0{n^). 

Proof . We first observe that every list Sjj for < / < j < « contains at most 0(1) states, each of 
which can be represented by one integer of size respectively 0(1) or 0{n) according to wheather 
G is finitely ambiguous or not. Since the space taken by the algorithm is essentially due to the 
memory required by the table Sij, we obtain a space complexity 0{n^) for finitely ambiguous 
grammars and 0{n^) in the general case. 

As far as the time complexity is concerned, note that in each loop, for a fixed I < j <n, 
the Scanner and Predictor phase execute 0{j) statements, while the Completer phase requires 
0{it) unit steps, where I is the maximum size of the sets Lg ,-, B G V, < / < 7 — 1. Now, for a 
general grammar, we have I = 0{j) which implies a total number of unit steps 0{n^): each of 
them requires logarithmic time 0{n) leading the overall time complexity to 0{n^). 

On the contrary, in the case of a finitely ambiguous grammar G, we have i = 0(1) and 
hence we only need C?(log?i) logarithmic time to locate each state in the table yielding a total 
time complexity 0{n^\ogn). I 



4.1.3 Inherently ambiguous context-free languages 

Now, let us go back to our original problem and let L C £* be a c.f. language. We recall that 
L is unambiguous if it is generated by an unambiguous c.f. grammar, while it is inherently 
ambiguous whenever every c.f. grammar G generating L is ambiguous. We also say that L 
is finitely (respectively, polynomially) ambiguous if it is generated by a finitely (respectively 
polynomially) ambiguous c.f. grammar. 

Observe that there are natural examples of polynomially ambiguous c.f. languages which 
are not finitely ambiguous. For instance, if L = \ww ^ : w € |a,Z7||, then it turns out that 



is inherently ambiguous, but not finitely ambiguous (IHarrisonL I1978L Section 7.3): however, 
it is easy to verify that is generated by a grammar of ambiguity 0{n) given by the set of 
productions S^AB, A^aAa\bAb\z and B^aBa\bBb\e. Similarly, for every /: G N, the language 
(L^ • {0})*^ is generated by a grammar of ambiguity 0{n'^). 

Then, applying Proposition j4. 1 . 1 l and Theorem 14. 1.1 I to Theorems 13.2. Il and l3.2.2[ we obtain 



Theorem 4.1.2 IfL is a polynomially ambiguous context-free language, then there exists a poly- 
nomial time u.r.g. for L and a fully polynomial time ra.s. for its census function Ci. If m^oreover 
Cl is polynomially bounded, then there exists a polynomial time re.c.for L. 

In particular, in the case of finite ambiguity, one gets the following 

Corollary 4.1.1 IfL is a finitely ambiguous context-free language, then there exists a u.r.g. for 
L working in 0{n^\ogn) time on a PrRAM under logarithmic cost criterion and a ra.s. for its 
census function Ci of the same time complexity. 
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4.1.4 The grammar as a part of the input 

Observe that the results of Sections [4.1.1 l and 14. 1 . 21 assume as fixed a c.f. grammar G. In view of 
an application presented in the next section, here we study the complexity of similar algorithms 
taking G as a part of the input. 

To this end, we need a definition of the dimension of a c.f. grammar; if G = (y,r,5',P), we 
define its dimension |G| as the number of bits needed to encode it, which is 0(#P£Glog(#V + 
#£)), where £g is the maximum number of symbols appearing in the right hand side of a pro- 
duction in P. Also in th is case, we restrict to c.f. grammars in Chomsky normal form, since it is 
known ( Harrison . 19781) that every c.f. grammar G can be transformed in a c.f. grammar G' in 



Chomsky normal form in time polynomial in |G| (and hence is polynomially related to |G|). 

Given a c.f. grammar G = {¥,11,5,?) (in Chomsky normal form) and n > 0, what is essen- 
tially needed to generate (uniformly at random) a derivation tree of a word of length n, are the 
coefficients Ca(^), for 1 < ^ < « and A € V ( see Section l4.1.1l) . Observe that the method used in 
that section, fundamentally based on Comtet's recurrence equation, can, in gen eral, requ i re tirn e 
exponential in \G\. Nonetheless, by a modified version of the CYK algorithm ( Harrison . 19781) . 
we obtain the following 

Lemma 4.1.2 There exists an algorithm that, having as input a c.f. grammar G = (V, Z, 5, P) ( in 
Chomsky normal form), A G V and n > 0, computes Ca(«) in 0{n^\G\) time on a RAM under 
logarithmic cost criterion. 

Proof . We design such an algorithm by an apphcation of dynamic programming and of the well 
known convolution property 

CA{i)= I 'tcB{i)Cc{i-i) 

A^BCePi=l 

which holds for every A G V and £ > 0. Consider Algorithm 14.31 where c{A;k), for A and 
1 </:<«, is an array of integer. 



1 inputG = {V,L,S,P),A,n 

2 forallBGV doc(B;l) ^0 

3 forallB^a ePdoc(B;l) ^c(B;l) + l 

4 for k = 2...ndo 

5 for all B^CD € P do 

6 c{B;k) ^ c{B;k) + '£Q^i^,,c{C;i)c{D;k-i) 

7 output c (A; «). 



Algorithm 4.3: Counting derivation trees. 

To prove the correctness of the algorithm, we prove a somehow stronger result: at the be- 
ginning of each iteration of the loop at lineHJ c{B;£) = CB{t) for every S G V and \ < I <k. 
By induction on k: ii k = 2, then the loop at lines [2] and [3] compute exactly Ca(1) for every 
B € y, moreover, the loop at line |4] makes no iteration, hence the statement is true. \f k > 2, 
then Y,o<i<k'^{C'4)c{D;k — i), by inductive hypothesis, is equal to the number of parse trees in 
Tg with the first derivation corresponding to B^CD. Then the loop at line [5] collects in c{B;k) 
the value Cb(/c) by iteratively summing over all the production in P with B on the left, for every 
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B S y. To obtain an upper bound on the computation time, observe that the sum of line |6] has 
at most n summands and is in a nested loop which is executed 0{n#P) times, and that all the 
involved integers have size 0{n) bits. I 

Then, as discussed in Section 14.1.11 we can apply Algorithms 14.31 and 14.11 to obtain the 
following 

Proposition 4.1.2 There exists a u.r.g. of derivation trees that, given a c.f. grammar G in Chom- 
sky normal form and n> as input, works on a PrRAM in time polynomial in \G\ and n. 

Furthermore, the algorithm of Section 14.1.21 to compute the number of derivation trees of a 
terminal string, can be used also assuming the grammar as part of the input. A simple analysis 
leads to the following 

Proposition 4.1.3 The number of derivation trees of a terminal string x in a context-free gram- 
mar G in Chomsky normal form can be computed in time polynomial in \G\ and \x\ on a RAM 
under logarithmic cost criterion. 



4.2 One-way Nondeterministic Auxiliary Pushdown Automata 

In this section we show how to apply the general technique of Chapter [3] to the case of lan- 
guages accepted by polynomial time one-way nondeterministic auxiliary pushdown automata 
(1-NAuxPDA, see Section 1231 for a definition of the model). 

We recall that, given a 1-NAuxPDA M, the ambiguity of M is the function t/^ : N — ?> N 
defined as the maximum number of accepting computations for every input string of length 
« € N. Hence, we say that M is polynomially ambiguous if, for some polynomial p{n), we have 
dmin) < p{n) for every « > 0. We also recall that it is well known that, given an integer input 
n > 0, a context-free grammar G„ can be built, in time polynomial in n, such that L{G„) fl £" = 
L(M) nZ", where L{G„) C £* is the language generated by G,, and L{M) C £* is the language 
accepted by M. This allows us to apply the results of the previous section to the languages 
accepted by 1-NAuxPDA. 



Here, we describe a modified version of the usual construction of G„ given by lCookl(ll97lh : 



AUender. Bruschi. and Pighizzinil (1 19931) which allows to bound the ambiguity of G„ with re- 



spect to the ambiguity of M. First of all, we assume without loss of generality that the automaton 
cannot simultaneously consume input and modify the content of the stack and that at most one 
symbol can be pushed or popped for each single move. 



A surface configuration dCookl 119701) of a 1-NAuxPDA M on input a: G E+ of length n is 
a 5-tuple {q,w,i,rj) where q is the state of M, w the content of its work tape, 1 < / < [wj the 
work tape head position, F the symbol on top of the stack and 1 < 7 < n + 1 the input tape head 
position. Observe that there are n'^^^^ surface configurations on any input of length n>0. Two 
surface configurations C\,C2 form a realizable pair {C\,C2) (on a wordy G Z+j iff M can move 
(consuming input y) from Ci to C2, ending with its stack at the same height as in Ci, without 
popping below this height at any step of the computation. If {C\ , D) is a realizable pair on y' and 
(Z),C2) is a realizable pair on y" , then it is straightforward to check that (Ci,C2) is a realizable 
pair ony = y'y" . Let ^„ be the set of surface configurations of M on inputs of length n and 
define the c.f. grammar Gn{M) by the following statements: 
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(a) the set of nonterminal symbol V contains each (Ci , C2 , ^) such that Ci , C2 G and £ G 

{0,1}; 

(b) the set of terminal symbols is E; 

(c) the initial variable is (Cin,Cfin), where Qn and Cfin represent respectively the initial and final 
surface configuration of M; 

(d) the set P of productions is given as follows: 

(1) (Ci,C2,0)— T-a € Z' iff (CijCi) is a realizable pair on a G ZU {s} via a single move 
computation; 

(2) (Ci,C2,0)^(Ci ,D, 1)(D,C2,^) G P, for £ G {0, 1}, iff Ci,C2,D G 

(3) {Ci,C2,l)^{Di,D2,£) G P, for £ G {0,1}, iff Ci,Di,D2,C2 G Dj can be reached 
from C\ by a single move pushing a symbol on top of the stack and C2 can be reached 
from D2 by a single move popping the same symbol from the top of the stack. 



For the sake of brevity define, for each realizable pair (Ci,C2), the set ^(CijCj) of all 
computations of M starting from Ci and ending in C2 with the stack at the same height as 
in Ci, without popping below this height at any point of the computation; define also the 
subset '^i{Ci,C2) ^ '^(Ci,C2) of such computations (of at least two steps) during which the 
stack height never equals the stack height at the extremes C\,C2 and the subset '^^(Ci,C2) = 
'io{C\,C2) \ '^\{C\,C2) of computations during which the stack height equals, at least once, the 
stack height at the extremes C\,C2- 



Fist of all, observe that, following ICookl (Il97lh and lAUender et al.l (1 19931) . it is possible to 
prove the 



Proposition 4.2.1 Given a 1-NAuxPDA M working in polynomial time, there exists an algo- 
rithm computing, on input n > 0, the c.f. grammar Gn{M) in polynomial time on a RAM under 
logarithmic cost criterion. 

We now turn to show that the c.f. grammar Gn{M) defined above exactly generates the set 
of strings of length n accepted by M: 

Lemma 4.2.1 Given a 1-NAuxPDA M, the c.f. grammar Gn{M) is such that 

( i) if {C[ , C2) is a realizable pair on y via a computation in 'We (Ci , C2 ), then (Ci ,C2,£) ^ y; 

(ii) if {C\,C2,£) then (Ci,C2) is a realizable pair on y via a computation in'i§'({C\,C2). 

Proof . We prove statement % by induction on the number m of computation steps from C\ to 
C2. If m = 1 then the computation can only be in 'Wq{C\,C2) and consumes at most one symbol 
aGru{£};then, by (Ci,C2,0)^a 4> a. 

If m > 1 and the computation is in ^o(Ci,C2) let D be the first surface configuration after 
C\ in the computation for which the stack height equals that of C\,C2', it is then straightfor- 
ward to check that (Ci,D) is a realizable pair on some y' via a computation in '^i(Ci,D) and 
(€2,0) is a realizable pair on some /' via a computation in 'rf({D,C2), for some £ G {0, 1}, 
where y = y'y"; moreover these computations have both fewer steps then the one between 
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C\ and C2. Hence, by inductive hypothesis, (Ci,D,l) ^ y' and (D,C2,^) =4> and, by (Id2l ). 
(Ci,C2,0)^(Ci,D,l)(D,C2,£)^y/=3'. 

Finally, if m > 1 and the computation is in '^1 (Ci ,C2), then it is straightforward to check that 
the first move after C\ (say, to a surface configuration DO has to be a push and the last move 
before C2 (say, from a surface configuration D2), has to be a pop of the same symbol; moreover, 
since the moves from C\ to Di and from D2 to C2 alter the stack and hence consume no input, 
{D\,D2) is a realizable pair ony via a computation in 'io({D\,D2), for some I E {0, 1}, have fewer 
steps than the computation between C\ and C2. Then, by inductive hypothesis, {D[,D2,£) y 
and, by ddSjl, {Ci,C2,l)^{Di,D2,£) ^ y. 

Now we prove statement dull by induction on the number k of derivation steps. If ^ = 1 the 
only production leading to a terminal symbol is the one in (IdTI ). hence the statement is obvious. 

On the other hand, if ^ > 1 and £ = 0, then, by (Id2l ). the derivation can only be of the form 
(Ci,C2,0)^(Ci,D,l)(D,C2,f) 4>y/ =y, for some £' G {0,1}, where (Ci,D,l) ^ y' and 
(D, C2 , £') ^ y" and both derivations have less then k steps; then, by inductive hypothesis, (Ci , D) 
is a realizable pair on y' and (£),C2) is a realizable pair on y", hence (Ci ,C2) is a realizable pair 
on y via a computation in '^o(Ci ,C2). 

If ^ > 1 and ^ = 1, then, by (Id3] ). the derivation can only be of the form (Ci,C2,l)^ 
{Di,D2,£') ^y, for some f G {0,1}, where (Di,D2,f) =^ y in k — \ steps; by inductive hy- 
pothesis, (Di ,D2) is a realizable pair on y; moreover, by (Id3] ). Di can be reached from Ci with a 
single push move and C2 can be reached from D2 with a single pop of the same symbol, hence it 
is straightforward to verify that (Ci , C2 ) is a realizable pair on 3^ via a computation in "^i (Ci , C2 ) . 
I 

In order to apply the result on c.f. grammars of Section 141] in this case, we have to bound 
the ambiguity of G„(M). To this end, we have the following 

Lemma 4.2.2 Given a 1-NAuxPDA M, the number of leftmost derivations (Ci,C2,^) ^ y in 
G„{M) is less than or equal to the number of computations in ^^(Ci,C2) consuming y. 

Proof . We want to prove that, for every C\,C2 € S^n and j G £*, the proofs of Lemma 14.2.11 
define a bijective map from the leftmost derivations {C\,C2,£) y to the computations in 
^f(Ci,C2) consuming y. 

By the construction, it is clear that such a map is surjective. Hence, we simply have to show 
that it is injective, i.e., if the computations associated with two leftmost derivations are equal, 
then the two derivations themselves are the same. We work by induction on the number m of 
steps of the computation. If m = 1 the statement is obvious. 

Ifm> 1 and£ = 0, let, for/G {1,2}, (Ci,C2,0)^'(Ci,D,-, 1)(D,-,C2,£,) ^jbe two leftmost 
derivations (for some G {0, 1} and G S^n)- By construction of the grammar, if the associated 
computations are equal, then D\ = D2, £\ = £2 and there exist two words y\y" such that y = y'y", 
(Ci,D;, 1) 4>y and {Di,C2,£i) for / G {1,2}. Then, by inductive hypothesis, (Ci,D,-, 1) ^ 
y' are the same leftmost derivation for / G {1,2} and (D;,C2,^,) ^ /' are the same leftmost 
derivation for / G {1,2}. Hence {Ci,C2,0)—?'{Ci,Di,\){Di,C2,£i) ^ y are the same leftmost 
derivation for / G {1,2}. 

If/M> 1 and£= 1, let, for/ G {1,2}, (Ci,C2, l)->(Di,;,D2,i,^,) 4>ybe two leftmost deriva- 
tions (for some £{ G {0, 1} and Di i,D2 i G If the associated computations are equal, then 
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again by the construction of the grammar, Di j = D12, D2,i = D22 and = I2, hence, for 
/ G {1,2}, {Dij,D2j,ii) =4> J. Moreover, by inductive hypothesis, the two derivations are the 
same leftmost derivation for / E {1,2}, hence (Ci,C2, \)—^{Dij,D2j,£i) ^ y are the same left- 
most derivation for / G {1,2}. I 



We can now conclude our discussion: by Theorem 13.2.11 and 13.2.21 an application of the 
results of Section 14.1.4 1 and of this section, leads to the following 

Theorem 4.2.1 Let L be the language accepted by a polynomial time 1-NAuxPDA with poly- 
nomial ambiguity. Then there exists a polynomial time u.r.g. for L and a fully polynomial time 
ra.s. for its census function Cl- If moreover Cl is polynomially bounded, then there exists a 
polynomial time re.c. for L. 



4.3 Rational Trace Languages 

Another application concerns the unifor m random generation and th e census function of trace 
languages. To study this case we refer to IPiekert and Metivied (119970 for basic definitions. We 
only recall that, given a trace monoid M(r,/) over the independence alphabet (£,/), a trace 
language, i.e., a subset of M(r,/), is usually specified by considering a string language L C £* 
and taking the closure [L] = {t £ M(r,/) : f = [jc] for some x € L}. In particular, a trace language 
T C M(r,/) is called rational if T = [L] for some regular language L C £*. In this case we say 
that T is represented by L and the ambiguity of this representation is the function : N ^ 
N, defined by diin) = maX;te5;« #(L n [x]). We say that a rational trace language T is finitely 
ambiguous if it is represented by a regular language L such that, for some ^ e N, diin) < k for 
every « > 0; in this case we say that T has ambiguity k. 

In the following, assuming a given independence alphabet (S,/), we denote by M the set of 
all rational trace languages in M(r,/) and, by ^j^, the subset of trace languages in M of ambigu- 
ity k. Clearly, for every independence alphabet (£,/), we have ^1 C M2 C • • • C IJ^^j ^ M. 



Th e properties of these families of languages h ave been studie d in the literature by lBertoni et al 
(1982); Bertoni. Mauri, and Sabadini ( 1985); Saka rovitchI ( 1987 ) extending a previous study on 
unamb iguous rational sets in totally commutative monoids by Eilenberg and Schiitzenberger 
(119691). In particular, it is kno\yn that ^\ = iff the independence relation / is transitive 



(IBertoni et al.Ul985l ; ISakarovitchlll987r) ; on the other hand, if / is not transitive, then we get the 



following chain of strict inclusions: 



?i C ^2 £ • • • £ U -^-t £ ■ 



We say that a rational trace language T is polynomially ambiguous if it is represented by a 
regular language L such that, for some polynomial p{n), we have di,(n) < p{n) for every « > 0. 
Observe that there exist examples of polynomially ambiguous rational trace languages that are 
not finitely ambiguous. For instance, fixing I = \(a,b), {b,c)\, if L = {a* c)* {ab)* c{a* c)* , then it 
turns out that [L] does not belong to Ur=i (IBertoni et al.Lll982l) : however, [L] is polynomially 
ambiguous with dL{n) = 0{n) since, for every x of the form x = a^^ca^^c . . . {ab)'^' ca'^'+^ 
with k\,...,ks,...kt G N, it holds 



c. . .a^'c 



L n [x] = {a^' c... (abf'c ...a^'c:ki = k„ l<i<t}. 
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Now, let us go back to our problem: here, we want to use L as an ambiguous description 
of [L]. Also in this case, we first have to design two routines: one for generating a word in L 
uniformly at random and the other for determining the number of representatives of a trace in 
a given regular language L. The first routine is already discussed in Section 13.1.41 The other 
algorithm is ob tained by adapting a procedure for solving the membership problem for rational 
trace language ( Bertoni. Mauri, and Sabadini . 19891 : Avellone and Goldwurm . 1998 ): 



Proposition 4.3.1 Given an independence alphabet (^,1) and a regular language L C £*, the 
problem of determining the cardinality ofL [x], given x^Las input, can be solved on a RAM 
under unit cost criterion in 0{n'^) time, where (X is the maximum clique size in (£,/). 



As a consequence, we can apply the general paradigm for uniform random generation and 
approximate counting presented in Chapter |3l This leads to the following 



Theorem 4.3.1 Let T C M(r,/) be a finitely ambiguous rational trace language and assume 
1^0. Then, T admits a u.r.g. working in 0{n'^\ogn) time and using 0{n^\ogn) random bits on 
a PrRAM under logarithmic cost criterion, where CH is the size of the maximum clique in (£,/). 
Moreover, there exists a r.a.s. for the census function ofT of the same time complexity. On the 
other hand, ifT is polynomially ambiguous, then it admits a polynomial time u.rg. and a fully 
polynomial time ra.s.for its census function. 
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Spesso c 'e bonaccia sulla pagina. 

Inutile girarla per cercare 

I 'angolo del vento. 

Si stafemii, 

il pensiero oscilla, 

si riparano le cose 

che la navigazione ha guastato. 

Valeric Magrelli, Ora serrata retinae 



In this brief chapter we investigate how to extend some result of the two previous chap- 
ters about combinatorial structures back to the more general case of /^-relations. In particular, 
we shortly investigate some closure property of some classes of p-relations under union and 
complement boolean operators, once suitably adapted to the case of p-relations. 



5.1 From Combinatorial Structures to p-Relations 

As we have stated at the beginning of Chapter [3l for the sake of brevity and clarity in the def- 
initions and proofs, we have restricted our attention to combinatorial structures instead of p- 
relations. Here we want to briefly and informally suggest that the results obtained so far can be 
generalized back to the case of 79-relations. 

To this aim, one needs a definition of description suitable to the case of relation. A proposal 
can be the following: a p-relation 5 C £* x £* is an (ambiguous) description of a /^-relation 
/? C £* X £* via the function / : 5 — > /? iff , for every a, p G £* such that a /? P, there exists some 
Y G S(a) such that /(a, y) = (a, P). In this case, the ambiguity of the description is the function 
d -.R^N defined by d{{a, P)) = #{y € 5(a) : /(a, y) = (a, P)} and the description is said to be 
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polynomial whenever / and d are computable in time polynomial in |a| and there exists some 
D G N such that d{{a,^)) = 0(|a|^). 

The results of Chapter[3]can be rephrased according to the following 

Corollary 5,1.1 If a p-relation R has a polynomially (ambiguous) description S which admits a 
polynomial time u.r.g., then R admits a polynomial time u.r.g.; moreover, if, for every a G £*, the 
cardinality ofS{cC) can be computed in time polynomial in \(x\, then R admits a fully polynomial 
ra.s.. 

In particular, Theorem 12.3.11 of Section 14.21 can be generalized to extended one-way non- 
deterministic auxiliary pushdown automata (see Section 12.31) . obtaining the following stronger 
version of Corollary 12.3.11 

Corollary 5.1.2 If a p-relation /? C £* x £* is accepted by a polynomially ambiguous extended 
one-way nondeterministic auxiliary pushdown automaton working in polynomial time, then R 
admits a polynomial time uniform random generator 

Proof . The proof follows from an argument similar to the beginning of the proof of Theo- 
rem l2.3.l} given an extended 1-NAuxPDAM, with state set Q, and some a G £*, it is always pos- 
sible to build (in time polynomial in |a|) a 1-NAuxPDAM("\ with state set 2 x {1, . . . , |a|} x Z, 
such that the computation of M'"' on input p efficiently simulates the computation of A on input 
(a, P) for every P G £* and has the same ambiguity as M. The result then follows by applying 
Theorem |40 to M("). | 



5.2 The p -Union 

In this section we want to investigate the closure with respect to the union of some of the classes 
of /^-relations we have encountered in previous chapters. 

First of all, we need the following 

Definition 5.2.1 Let 5 C £* x I* and RCI,* xL* be two p-relations. Their p-union R(JP S is 
defined as a (RU'^ S) ^ iff (X R ^ or a S p, whenever the polynomials which relate the length of 
the elements in the domain to the length of the elements in the codomain are identical, otherwise 
is undefined. 
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Then, by means of Corollary 15.1.11 and Theorem l3.3.2[ one can obtain the following 

Proposition 5.2.1 Let 5 C £* x £* and /? C Z* x £* are two p-relations such that R U'' S is 
defined. Ifru^ FP and rg € FP, then RU'^ S admits a polynomial time u.r.g.. 



On the other hand, it is known (lHuvnhl . [l99(i Theorem 3.1) that there exist two unambigi 



uous 



context free languages Li,L2 such that their union is inherently ambiguous an d its rank is not 



polyn omial time computable unless P = P^*^. Moreover, it is also known (Goldberg and Sipsen. 



19911) that unambiguous context free languages are rankable in polynomial time and hence the 



slice relation^ of Li,L2 are both rankable in polynomial time. 
We can summarize this discussion in the following 

Proposition 5.2.2 The class of p-relations that are rankable in polynomial time is not close 
under p-union, hence it is a proper subclass of the class of p-relations admitting a polynomial 
time u.r.g. (unless P = P'^^j. 



5.3 The p -Complement 

Now we consider the closure with respect to the complement of some of the classes of p- 
relations. For this reason, we suggest the following 

Definition 5.3.1 Given a p-relation /? C £* x £*, its p-complement R^'^ is the p-relation RP'^ C 
I* x £* defined asaRP' p iff not aR^ and |p| = p{\a\). 

Now, we show that there exists a /^-relation R such that it admits a polynomial time u.r.g., 
but the same does not hold for its /j-complement. Let R = {G,U) be the /j-relation such that 
[/ C V is an independent set of the graph G = {V,E), i.e., for every pair of nodes u,v £ U the 
edge (m,v) does not belong to £"; if V = {vi, . . . ,v„}, assume to represent U over {0, 1}* by a 
string u\ .. .Un such that m,- is 1 iff Vj £U, for I <i <n. 

It is then clear that RP'^ is the relation {G,U) where either G is not (a correct encoding of) 
a graph and U is any string of length p{\G\), or G = {V,E) is a graph and U is such that 
(m,v) € E for some u,v gU. 



As proved by iSinclairi (I1988L Theorem 1.17), if R admits a polynomial time u.r.g., then 
NP = RP. However, using the algorithm for generating uniformly at ra ndom the sa t isfyin g 
assignment to a boolean formula in disjunctive normal form (DNF) given by ljerrum et al. ( 19861) . 
we can prove the following 

Lemma 5.3.1 There exists a polynomial time u. rg. for RP'^. 

Proof . Let G be an element of the domain of R. If G is not a graph, it is easy to generate 
uniformly at random an element of {0, 1 . On the other hand, let G = {V, E) be a graph and 
consider the DNF 

\J XyNXyi 
(vy)eE 



see Section[L2l 
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in the variables Xq = {xy : v G V}. It is easy to verify that every truth assignment x :Xg ^ 
true, false} satisfies the DNF iff U = {v : x{x^) = true} i s such that {G,U) € RP'^. Hence, the 
result follows by applying the u.r.g. for the DNF given by iJerrum et al. (Il986h . I 



On the other hand, by observing that r^pc (a, P) = #{y : y ^lex P} — P), one can imme- 
diately conclude the following 



Proposition 5.3.1 The class of p-relation that are rankable in polynomial time is closed under 
p-complement, hence it is a proper subclass of the class of p-relations admitting a polynomial 
time u.r.g. (unless NP = RPj. 

The situation discussed in these last two section it then illustrated by Figure 15.11 where the 
endpoints of the dotted arrows are as shown (from top to bottom) if NP ^ RP and P ^ . 



admitting a polynomial time uniform random generator 



RPC- 




Figure 5.1: Some classes of p-relations. 



Chapter 6 



Applications to 
Combinatorial Optimization 



Hay que cansar los numeros. 

Que cuenten sin parar, 

que se embriaguen contando, 

y que no sepan ya 

cudl de ellos sera el ultimo: 

I que vivir sin final! 

Que un gran tropel de ceros 

asalte nuestras dichas 

esbeltas, al pasar, 

y las lleve a su cima. 

Que se rompan las cifras, 

sin poder calcular 

ni el tiempo ni los besos. 

Pedro Salinas, La voz a ti debida 



In this chapter we discuss an application of uniform random generation to combinatorial 
optimization; notice that the research field of combinatorial optimization is so relevant to the 
theory and practice of computer science that part of the contents of this chapter has been the 
very inspiration originally motivating this entire work. 

After a short general introduction, we focus on the local search paradigm which is probably 
the most used technique to solve combinatorial optimization probl ems sin c e it co mbines sim- 
plicity and empirical success in many applications. As studied by iGrossi (Il999h . one way to 



improve and fully exploit the power of local search consist in the use of uniform random gen- 
erators for producing an initial solution; in connection to this fact, it has been possible to define 
the class of expectation-guaranteed problems: a wide and natural class whose member admit 
efficient deterministic approximation algorithms obtained by derandomizing such a process of 
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random generation. In the last section of this chapter we show a negative result concerning this 
approach by proving that not every problem admits an efficient derandomization, unless P = jjP. 



6.1 Some Basic Definitions and Properties 



This section is intended to be a very short introduction to the basic definitions and known facts 
about combinatorial optimization; a complete and exhaustive treatment of this subject if far 
behind the scope of this work; for a more comprehensive treatment of t his topic, see for instanc e 
( Garey and Johnson . 19791 ; Bovet and Crescenzi . 1993 ; Papadimitriou . 1994 ; Hochbaum . 1997 ). 



6.1.1 The class NPO 



We begin by recalling the definition of a particular class of combinatorial optimization problems 
which captures the most part of problems of practical interest; we observe that our definition is 
slightly different from the standard one because of our use of /j-relations 



Definition 6.1.1 A NP optimization fNPOj problem is a 3-tuple (/?,g,goal) where RQL* xL* 
is a p-relation, the objective function g : £* x £* — N a polynomial time computable function 
and goal G {min,max}. Moreover, the problem is said to be polynomially bounded whenever 
the value ofg{a, P) is bounded by a polynomial in \a\, for every a, p. 

As we have anticipated in Section [TT2l the /^-relation R serves both to express, via its domain, 
the set of instances of the problem dom(/?) = {a € £* : a /? P for some p € £*} and, for each 
instance a G dom(/?), the set of its feasible solutions sol(a) = {p G £* : a 7? P}. The objective 
function, also called value, or measure of the solution, together with the goal of the problem, 
defines its optimum as 

opt(a) = goal g(a,p). 

Pesol(a) 

Hence, the solution of a NPO problem consists in finding, for every instance a G dom(/?), an 
optimum solution p G sol(a) such that g(a,p) = opt(a). Finally, recall that with every NPO 
problem it can be associated its decision version whose solution is to decide, for every pair (a, t) 
with a G dom(/?) and f G Q, whether opt(a) < t (if goal = min), or opt(a) > t (if goal = max). 

Some example of problems in this class are the "Max Cut" and "Max Clique" problem^ 
In both cases, the instances are graphs G = iy,E). For "Max Cut" case, for each graph G, 
the feasible solutions are partitions of its vertex set V into disjoint sets (^1,^2) and the goal 
is to maximize the measure of a partition, defined as the cardinality of the induced cut, i.e., 
the number of edges with one endpoint in V\ and the other in V2- On the other hand, in "Max 
Clique" problem, for each graph G, the feasible solutions are subset of vertices V' QV which 
are all pairwise adjacent, i.e., every pair of vertices in V' are joined by an edge in E; in this case 
the goal is to maximize the measure of such subsets, defined as their cardinality. 

'for a thorough compendium of NPO problems, see the survey of ICrescenzi and Kamil ( Il998t) . 
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6.1.2 Approximation algorithms and approximation classes 



It is well known ( Karpl 19721 : Garey and Johnson . 19791) that, under the conjecture P / NP, 
every NPO problem whose corresponding decision version is NP-complete is not solvable in 
polynomial time. As a consequence, if we are interested in "efficient" algorithms, we have 
to restrict our attention only to polynomial time computable "approximate solutions" of the 
problem. More formally 



Definition 6.1.2 Let {R,g, goal) be a NPO problem. Given an instance a G dom(/?) and a fea- 
sible solution P S sol (a), the performance ratio ^{a, p) o/P with respect to a is 

^(a,P)=maxj^,^ 
U(a,p)'opt(a) 

and the relative error S' (a, P) of P with respect to a is 

|opt(a)-g(a,p)| 



(a,P) 



max{opt(a),g(a,p)} 



Obviously, the closer to 1 (respectively 0) is the performance ratio (respectively relative error), 
the better are the solutions. There is a strict relationship between the performance ratio and the 
relative error: 

The concepts expressed so far allow us to define what an approximation algorithm is 

Definition 6.1.3 Given a function r : N — )• [1,°°), an algorithm A is a r(?i) -approximation algo- 
rithm/or a NPO problem (7?,^, goal) iff, for every instance CH G dom(/?), it returns a feasible 
solution A{a) G sol(a) such that 

^(a,A(a)) < r([a[). 



Similarly, one can define an approximation scheme as follows 



Definition 6.1.4 An algorithm A is an approximation scheme /or a NPO problem (/?,§, goal) ijf, 
for every instance a & dom{R) andr& i^i°°)> it returns a feasible solution A{a, r) € sol(a) such 
that 

,^{a,A{a,r)) < r. 

An approximation scheme is said to be polynomial if it works in time polynomial in |a| and fully 
polynomial if it works in time polynomial in |a| and r/{r— 1). 

This notion gives rise in a natural way to various classes of NPO problems according to the 
existence of approximation algorithms satisfying specific performance bound. As an example, 
we just recall some of such classes we will refer to later on in this chapter. The class of NPO 
problems approximable within a constant factor is defined as follows 



Definition 6.1.5 A NPO problem belongs to the class APX if, for some r > 1, it exists a polyno- 
mial time r-approximation algorithm for it. 



Then we can look for problems admitting efficient approximation schemes 
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Definition 6.1.6 A NPO problem belongs to the class PTAS if it admits a polynomial time ap- 
proximation scheme; similarly, it belongs to the class FPTAS if it admits a fully polynomial time 
approximation scheme. 



Observe that these classes do not trivially coincide since, for instance, it is known that every 
problems in NPO whose decision version is strongly NP-co mplete cannot have a fully polyno- 
mial time approximation scheme (iGarey and Johnsonl . ll979i) . As it is obvious to conclude from 
the previous definitions 



Proposition 6.1.1 FPTAS C PTAS C APX. 



6.1.3 Logically definable approximation classes 



Other classes of NPO problems c an be described by means o f the so called logical definabil- 
ity (for a survey, see the work of iKolaitis and Thakun (Il994r) '). a concept inspired by Fagin's 
characterization of NP in terms of second-order logic on finite structures (IFaginl. Il974[) . More 
precisely, according to such approach, a class of finite structures is NP-computable iff it is defin- 
able by an existential second-order formula, i.e., an expression of the form 3S(^{G,S), where G 
is & finite structure, S is a finite sequence of predicate symbols of bounded arity, and (j)(G,S) is a 
first-order formula. It is well known that every such formula is equivalent to a formula in prenex 
normal form, i.e., one of the form 3SNx3y^^{x,y,G,S), where cp is a quantifier-free formula and 
x= {x\,...,Xj),y = {y\,...,yk) are fi nite seqiiences of first-order variables (for an introduction 
to the logic details, see, for instance, (lBarwiselll977r) ). 



According to this framework, iPapadimitriou and YannakakisI (119911) introduced the class 
MaxNP of maximization problems whose optimum can be defined as 

max |{jc : 3yi^{x,y,G,S)}\, 
s 

where cp is a quantifier-free first order formula. Intuitively, in a NP decision problem one seeks 
the predicates 5 that satisft some existential second-order sentence Vjc3)'(p(jc,3',G,S), while in 
the corresponding maximization version in MaxNP one seeks those predicates S that maximize 
the number of tuples x satisfying the existential first-order sentence 3y(p{x,y,G,S). 



A canonical example in such class is the "Max Sat" proble the instances of which are 
collections C of clauses, i.e., a disjunction of literals, where a literal is a variable or a negated 
variable; feasible solutions are truth assignments to the variables and the goal is to maximize 
the number of clauses satisfied by the truth assignment. The optimum of such problem can be 
written as 

max 1{(0 e C : 3x[(r(;c) AP{03,x)) V {^T{x) AN{03,x))]}\ 

where T = {x ^ X : x(x) = 1} is the set of variables which are true under the truth assignment 
X : X — > {0, 1}, and the binary predicates P{(i),x) and N{(i),x) express the fact that the literal x 
appears as positive, respectively negative, in the clause CO. 



The syntactic view has proven useful not only in obtaining structural complexity results, but 
also in d eveloping paradigms for design i ng effi cient approximation algorithms as, for instance, 
stated by IPapadimitriou and YannakakisI (Il99lh . 



^For a more formal definition, see jCrescenzi and Karinlll998l) . 
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Proposition 6.1.2 For every problem in MaxNP there exists a polynomial time approximation 
algorithm whose performance ratio is bounded by a constant, therefore 

MaxNP C AFX. 

The same clearly holds for a subclass of MaxNP, called MaxSNP ("S" stands for strict), con- 
sisting of those maximization problems that are defined by quantifier-free formulae, such as 

max|{jc : (p(jit,G,5')}|, 

5 

where cp is quantifier-free. The class MaxSNP contains several natural maximization problems 
such as "Max 3-Sat" (the version of "Max Sat" with exactly three literals per clause). 



6.1.4 Approximation preserving reductions 

A basic notion in complexity theory is that of reducibility . Roughly speaking, a reduction be- 
tween two decision problems allows one to transforms every algorithm for solving a problem in 
an algorithm to solve the other. More formally, if the languages L,L' C £* encode two decision 
problems, we say that there is a many-one reduction from L to L' if there exists a polynomial 
time computable function /:!*—)•£* such that CO € L iff /(co) G L'. 

Unfortunately, in the context of approximation, these reductions do not guarantee to preserve 
the objective function, and in general, the quality of the solution. Thus, new approximation- 
preserving reductions have been introduced, i.e., reductions that not only map instances of one 
problem into instances of the other, but that are also able to map back "good" solutions into 
"paragona bly good" solutions. As an example of such id ea, we only recall the definition of AP- 



reduction (ICrescenzi. Kann. Protasi. and Trevisanl 119951) . that will be used later in this chapter. 
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Definition 6.1.7 A NPO problem (/?,g,goal) is said to be AP-reducible to a NPO problem 
(/?',g',goal') iff there exist two polynomial time computable functions f and b, and a constant £ 
such that, for every a € dom(/?), r > 1 and P' G sol'(/(a,r)), 

1. f{a,r) edom{R'); 

2. Z?(a,P',r) G sol(a); 

3. ^{f{a,r),^') < r implies M{a,b{a,^' ,r)) < l+£(r-l). 



Given a concept of reduction, it is possible to define the concepts of hardness, completeness 
and closure for a given approximation class of NPO problems as in the case of decision 
problems 

Definition 6.1.8 A NPO problem is said to be 'if -hard (under a given approximation preserving 
reduction) if all NPO problems in can be reduced to it; in addition, the problem is said to be 
"^-complete if it also belongs to The closure of^ under such reduction is the class of all 
NPO problems reducible to some problem in 



These ideas are widely studied, for instance byl Papadimitriou and Yannakakis (1991); Khanna. Motvyani. Sud 



(11994^ . We observe that one more reason of interest for th e study of the class M axNP is that the 
first example of a complete problem has been given for it (IKhanna et al.ll 1994b . 



Proposition 6.1.3 The NPO problem "Max Sat" is complete for MaxNP. 



6.2 Local Search 



In this section we introduce the basic idea of the local search paradigm with a discussion of some 
of its strengths and limits. Then we discuss in which sense randomization can help to improve 
such a paradigm. Here, for the sake of brevity, we focus only on maximization problems; a 
similar discussion can be easily carried out for the minimization case. 



6.2.1 The basic idea 

In this section, to simplify the notation, we restrict our attention to a very "simple version" of 
a combinatorial optimization problem. Let X be a discrete search space and let / : X ^ N be 
the goal function that must be maximized on a set of feasible solutions 5 C X. The local search 
paradigm is a general scheme to design approximation algorithms based on stepwise improve- 
ment on the value of the function / by exploring a "neighborhood" of the current solution. 

A neighborhood structure is a map S — > 2"^ that defines, for each x G 5, a set Sx C 5 of 
solutions that are, in some sense, "close" to x (we assume the map is such that j G 5^ iff x G Sy). 
The set Sx is then called the neighborhood of the solution x, and each y £ Sxis called neighbor 
of X. 

As an example, if X = {0, 1}", one can define, for every h > 0, the so called h-bounded 
neighborhood structure as the map x i-^ {y £ X : du{x,y) < h}, where J// : X x X — )• N is the 
Hamming distanc^ 

^We recall that d/f (x,>') is defined as the number of positions in which x differs from y. 



§6.2 



Local Search 



65 



Once a neighborhood structure is fixed, an example of the local search paradigm is given by 
the schema of Algorithm 16. 11 



input xq & S 
repeat 

pick y G Sx 

if f{y) > f{x) then ;c ^ j 
until f{y) < f{x) for all y^S^ 
output X. 



Algorithm 6.1: Local Search: schema of algorithm. 

With the term "schema" we refer to the fact that various methods can be used to implement 
the pick statement, selecting one neighbor of the current solution, such as choosing at random, 
following some predefined order, or taking the neighbor of maximum value. 

The fundamental concept in the analysis of a local search algorithm is that of local, as op- 
posed to global, optimality: ;c G 5 is a local maximum (with respect to a given neighborhood 
structure) iff f{x) > f{y) for all y G Sx while x G 5 is a global maximum iff f{x) > f{y) for all 
J G 5. It is then clear that one of the main drawbacks of the local search paradigm is that in 
general it offers algorithms that terminate in a local optimum and, unfortunately, in general it is 
not easy to estimate how far it is from a global one. 

There is a clear trade-off between the quality of the local optima and the neighborhood 
structure: the larger are the neighborhoods the better are the local optima, but it may be harder 
(slower) to compute them. Moreover, the quality of the local optima is also strictly related to the 
choice of the initial solution xq, usually, to reduce the sensibility to it, local search algorithms 
are repeated on many different initial solutions (and then the best output obtained is adopted). 
Whenever it is hard to come out with a feasible initial solution, clearly another trade-off happens 
between the number of initial solution considered and the quality of the final local optimum 
obtained. 

Therefore, designing a good local search algorithm has remained up to now an experimental 
art, even if some guiding principles and techniques have been identified. Nonetheless, as exper- 
imentally shown, the local search paradigm usually leads to fast algorithms that find reasonably 
good solutions in a reasonable amount of time. For this reason, they cover a wide application 
area ranging from mathematical problems, to graph problems, to logical problems and so on. 



6.2.2 Local search and NPO problems 



Thanks to its generality and its empirical success, the local search paradigm represents cer- 
tainly one of the most used approaches in approximating difficult optimization problems. The 
co mplexity of finding locally optimal solutions for th ose problems has been main ly investigated 
by Ijohnson. Papadimitriou. and YannakakisI (119881) : lAusiello and Protasil (119951) . showing the 
strengths and the limits of such a paradigm. 



In particular, lAusiello and Protasil (119951 ) introduce the class GLO of NPO problems that 
have guaranteed local optima: a maximization problem (/?,g,max) has guaranteed local optima 
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iff there exists an integer constant h such that the value of all local optima is at least 1/h times 
the value of a global optimum. If a problem has guaranteed local optima, then there exists 
an /j-bounded neighborhood structure such that, for every instance a € dom(/?), every local 
optimum P of a has the property that opt(a) < h ■ g{a, (3) (a similar definition can be given when 
goal = min). The fundamental assumption here is that, given an instance a of a (polynomially 
bounded) NPO problem, it is possible to determine whether a given solution P of a is a local 
optimum, and in the negative case, to find a better solution in its neighborhood in polynomial 
time. Various problem^ can be shown to belong to GLO, such as "Max Cut", "Max Sat", "Max 
3-Sat". Moreover, the class GLO is a subclass of APX, but the inclusion is proper as shown 
for instance by the problem "Max /:-GSat" which is well known to be appr oxima ble within a 
constant factor, but that does not have guaranteed local optima tAlimontii. Il996h . A further 
result is that GLO = APX, where the closure is taken with respect to the AP-reduction. 



6.3 The "Expectation-Guaranteed" Class 



Different techniques have been suggested to overcame some of the drawbacks of the local search 
paradigm. Some of these are, for instance, based on the modification of the neighborhood struc 



ture, or o f the goal function, such as in the non-oblivious local search ( Khan na et al.L 1 19941 : 



AlimontiL [l996 ) where a "non-oblivious" version of the goal function is suitably constructed 
allowing to escape from local optima. 



A completely different approach is taken by iGrossil (11999!) . which focus the attention on the 
choice of a "good" initial solution as a starting point for applying the standard local search. A 
first and simple idea is to generate uniformly at random some different feasible solutions then 
taking as initial solution that one with the best value. Hence, a second step toward a determin- 
istic approximation algorithm consist in investigating when it is possible to derandomize such 
initialization phase. 



6.3.1 Randomized choice of initial solutions 

We start by considering a very simple randomized algorithm. Let (/?,g,max) be a NPO prob- 
lem, and let A/^ > 1 be a parameter whose value will be determined in the following. Then, 
RadomSearch is Algorithm 16.21 



input a € dom(/?) 
for/= l,...,N do 

generate P G sol (a) uniformly at random 

bi ^ p 

p ^ argmax,g{i^ _^}g(a,^,) 
output p. 



Algorithm 6.2: RandomSearch. 

Since the focus of this section is on the approximation performance aspects of combina- 
torial optimization, the statement "generate P G sol (a) uniformly at random" will be simply 



^For a definition of such problems, see dCrescenzi and KaiinLll998h . 
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understood as a call to a subroutine which returns, on every call, an (independent and uniformly 
distributed) element P of sol(a). A discussion of the details concerning such subroutine, in the 
spirit of what is studied in the other chapters of this work, is deferred to the end of this section. 

A very simple analysis shows that this algorithm finds a solution whose value is "close" to 
the expected value of g{a, p) (when p is uniformly distributed on sol(a)) with high confidence. 
More formally, define 



then, by an application of Hoeffding's inequality (see Appendix [A]), one can easily obtain the 
following 

Proposition 6.3.1 For every < 5 < 1 and £ > 0, if N > l/le^ - log (1/5), the RandomSearch 
algorithm, on input a € dom(/?), gives in output P such that g{a, P) > E[g(a, •)] — £ • opt(a), 
with probability at least 1—8. 



Hence, if we restrict our attention to the class of problems for which the expectation of the 
goal function is a reasonable approximation of the optimum, the RandomSearch algorithm could 
be used as a "goo d" rand omized approximation algorithm. We restrict again our attention to the 



case goal = max; iGrossil (Il999i) introduces the following class 



Definition 6.3.1 A NPO problem (7?,^, max) is RS-good iff there exists a real constant p > 
such that, for every instance a € dom(/?), it holds E[g(a, •)] > p • opt(a). 



In view of the analysis of the RandomSearch algorithm, we can conclude the following 



Proposition 6.3.2 If a given NPO problem (/?,g,max) is RS-good, then, for every < 6 < 1 
and £ > 0, there exists a randomized approximation algorithm such that, on input a G dom(/?), 
returns a solution P G sol(a) such that, with probability at least 1—5, 

g(a,P) > (p-£)opt(a), 

where p> is some fixed constant independent of a. 



Observe that similar definitions and results can be given for the case goal = min. 
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Two solutions arise naturally. On one side, we can actually implement a uniform random 
generator for the /^-relation R characterizing the NPO problem which is RS-good, hence ob- 
taining a randomized approximation algorithm with a constant performance ratio. One of the 
motivation of this thesis has been to investigate which classes of relations (or combinatorial 
structures) admit efficient u.r.g. in order to define classes of NPO problem for which the ap- 
proach taken in this section can lead to efficient randomized approximation algorithm with a 
constant performance ratio. Thanks to the result of Chapter |2] and [3] (in particular, from Corol- 
lary [5lL2] and [Z2?l]l, one can hence conclude. 

Corollary 6.3.1 Let {R,g, goal) be a NPO problem that is RS-good and defined in terms of a 
p-relation that either 

(1) belongs to the class ^{s{n),i{n),d{n)) of p -relations recognized by extended Turing ma- 
chines with simultaneous complexity bound s{n) ■i{n) ■d[n) = 0{\ogn), or 

(2) is recognized by a polynomial time extended one-way nondeterministic auxiliary pushdown 
automaton with polynomial ambiguity. 

Then, (R,g, goal) admits a polynomial time randomized approximation algorithm with a con- 
stant performance ratio. 

On the other hand, in the next section, we see that for some NPO problems, under suitable 
hypotheses, it is possible to deterministically obtain a solution whose measure is as good as the 
expected value for it. For those problems, as discussed in the following, it will be possible to 
have (deterministic) approximation algorithm with a constant performance ratio. 



6.3.2 Derandomization: the EG class 

It is well known that, for every random variable X : H i-^ R, it always holds E[X] < supg,g^X((jo): 
this is the so called intemality property of the expectation. We want to exploit this fact in relation 
to the analysis of the RandomSearch algorithm, considering the case in which it is possible to 
efficiently (and deterministically) compute a p for which g{a, p) is greater than or equa l to the 



expectation. More formally and restricting again our attention to maximization problems, iGrossi 
introduces the following class 

Definition 6.3.2 A NPO problem (/?,g,max) is RS-derandomizable iff there exists a polynomial 
time algorithm that, having in input an instance a G dom(/?), outputs a solution p G sol(a) such 
that g{a,^)>E[g{a,-)]. 



Then, by joining this property with the RS-good one. IGrossi (Il999h defines a class of prob 



lem for which there exists a good deterministic approximation algorithm 

Definition 6.3.3 The Expectation-Guaranteed class (EG) is the class of NPO problems that are 
both RS-good and RS-derandomizable. 

It is straightforward to see that, if (7?,^, max) is in EG, there exist a constant p > and a 
polynomial time algorithm A such that, for every a G dom(/?). 



g(a, A(a)) > E[g{a, •)]>?• opt(a) 



§6.3 



The "Expectation-Guaranteed" Class 



69 



where the first inequality follows since max) is RS-derandomizable and the second since 
it is RS-good. Hence, since similar definitions and results can be given for the case goal = max, 
we have the following 



Proposition 6.3.3 EG C APX. 

As proved by Grossi ( 1999 ). several natural NPO problem^ belong to this class, 



Proposition 6.3.4 The NPO problems: "Max Sat", "Max k-CSP", "Max k-Cut", "Max Bisec- 
tion ", "Min TSP- (1,2)" and "Max TSP- (1,2)" all belong to EG. 



In particular, as a consequence of the f act that every problem in MaxSNP can be viewed (for 
a suitable k) as a "Max /c-CSP" problem ( Khanna et all 1994) . and by the previously recalled 
proposition, one can conclude that 



Proposition 6.3.5 MaxSNP C EG. 



To conclude this subsection, we recall that lGrossil (119991) has also proven the following 



Proposition 6.3.6 The inclusions MaxSNP C EG C APX are strict. Moreover, EG = APX, 
where the closure is taken with respect to the AP-reduction. 



6.3.3 The method of conditional expectation 



In this subsection we discuss the method of conditional expectation (lErdos and Spenceil. 119741) : 
a particular technique which allows to show that some NPO problem, under suitable hypotheses, 
is RS-derandomizable. 

Let (7?,^, max) be a NPO problem over some alphabet Z, where 7? is a j)-relation. For every 
instance a G dom(/?) and 1 <k<p{\a\), define the subset of feasible solutions sol(a; b\, . . . ,bk) = 
{Pi . . . Pp(|a|) G sol(a) : Py = bj for I < j < k} having prefix bi,...,bk G £. Then, the expec- 
tation of g(a,P) when P is uniformly distributed on sol(a), conditionally to P having prefix 
b\,. . . ,bu G £, can be defined as 



By computing such conditional expectation on prefixes of increasing length, one can incre- 
mentally compute the P for which g{cL, P) is greater than or equal to E[g(a, •)], selecting for each 
"position" of the prefix the best possible value. More formally, consider Algorithm (63] 

By the properties of conditional expectatior^, one can verify that, for every bi,...,bk G S, 
E[g{a,-) I bi,...,bk] < maxE[g(a,-) | bi,. . . ,bk,b]; 

hence, a simple analysis shows that, if P is the output of the previous algorithm on input a, then 
g(a,P)>E[g(a,-)]. 

^For a definition of such problems, see jCrescenzi and Kami 1 19981) . 

^for a detailed discussion of the probabilistic aspects of such method, see Appendix lA.2.31 
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input a e dom(/?) 
fork= l,...,p{\a\) do 

at ^ argmax,,gj;E[g(a, ■) \ a^,. . . ,ak-i,b] 

output p. 



Algorithm 6.3: Conditional expectation. 

Let now T{n,k) be the maximum time required, by a suitable algorithm, to compute E[g{a, •) | 
bi,...,bk\ for all a S dom(/?) such that |a| = n. Then, since the argmax is computed over the al- 
phabet £ that is finite by definition, the computation time of the above algorithm is T{n,k)) 
Hence, we can conclude with the following sufficient condition for establishing whether a NPO 
problem is RS-derandomizable (Grossi, 1999) 

Proposition 6.3.7 If a NPO problem {R,g, goal) is such thatE[g{a, ■) \bi,...,bk]is computable 
in time polynomial in |a| and k, then {R,g, goal) is RS-derandomizable. 



When all solutions are feasible 

Now we investigate, in the particular case when all solutions are feasible, a sufficient condi- 
tion for establishing that E[^(a, •) \ bi,...,bi(] is computable in polynomial time. Assume for 
simplicit>|3] L = {0,1} and, as already stated, consider a NPO problem {R,g, goal) such that 
sol(a) = £''(1"!), for all a G dom(/?) (where p is the polynomial for which /? is a p-relation). Fix 
now some a G dom(/?) and let n = p{\a\); the goal function g can be written as 

g(a,Pi...p„)= I g(a,Xi,...,X„)P^---P^ = Ga(Pi,...,(3,0 (6.1) 

where Ga{^i, ■ ■ ■ is a «-th variate polynomial over {0, 1}" whose variables have degree at 
most one (and g{a,'ki, . . . are considered as coefficient in N). 

In the next section a more detailed discussion on polynomials and on their relation with 
pseudo-boolean functions will be carried on. Here, instead, we only want to sketch how this 
simple idea can lead to compute E[g{a, ■) \ bi,... ,bk\. First of all, recall that the expectation is a 
linear functional, such a relevant property that Erdos ironically used to call it "Adam's theorem"! 
Moreover, since the expectation is taken with respect to the uniform distribution on P G Z", we 
can actually consider the P, as "independent". From this very simple observations, it follows 
that 

E[g(a,-) |^i,...,^^]=E[Ga(|3i,...,(3„) 

gia,h...Xn)E[f^\'---^l"\bu...,bk] 

{h--Me{o,\}" 

£ g{a,h...X,,)bY---bl^ n 

(Xi,...A„)e{o,i}" j=k+i 

n—k 



£ g{aM.-.K)b\'---bl'(\) 



(X,....A„)e{0,l}" 
Ga{bi,...MM2,...M2). 



^Observe that this is not a limitation, since one can always encode over {0, 1} every finite alphabet £. 
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We can summarize this discussion by the following ( Grossi . 1999h 



Proposition 6.3.8 Let {R,g, goal) be a NPO problem on £ = {0, 1} such that sol(a) = Z^d"!) 
for every a € dom(/?) and let Ga be defined as in equation d6.iD . If there exists an algorithm 
that, for every a € dom(/?), evaluates Ga on {0,1,1/2}^^"!' in time polynomial in |a|, then 
(/?,g,goal) is RS-derandomizable. 



6.3.4 Derandomization and the EG class 



As suggested by lGrossi if a NPO problem is in EG, then by combining derandomization 

and local search, one can obtain an efficient approximation algorithm. As an example, assuming 
the condition of Proposition [6331 for a given neighborhood structure P i— )• (see Section [6.2. lb . 
we can have Algorithm [ 



input a G dom(/?) 
Step 1 

for ^ = 1, . . .,p{\a\) do 

at ^ argmax^g|o,i} <^a(ai, • • ■ ,ak-i,b, 1/2, 1/2) 
P^«i---fl'p{|a|) 

Step 2 
repeat 

pick P' € 5p 

ifg(a,p') >5(a,P) then [3 ^p' 
until /(p')</(P) for all p' G 5p 

output p. 



Algorithm 6.4: An approximation algorithm for EG problems. 

Following this idea, but in a more general fashion, Grossi ()l999') proved the existence of a 
"universal" schema of algorithm for all the problems in EG; more formally 

Proposition 6.3.9 For every NPO problem (7?,^, goal) in EG, (with respect to a suitable h- 
bounded neighborhood) there exists a p-approximation algorithm for (R,g, goal) where 

opt(a) 

sup 



aedom(i?)E[^(a, •)] 



This schema of algorithm is based on the computation of E[g(a, •) \ b\,. .. ,bk], hence, in order 
to obtain polynomial time algorithms following such a schema, E[g(a, •) \b\,. . . ,bk] should be 
computable in polynomial time. 



It is then clear that, in order to apply the derandomized initialization phase to local search 
algorithm as discussed in this section, it is of fundamental importance to determine whether 
E[g(a, •) I ^1 , . . . ,bk\, or Ga(ai, . . . ,ak-\,b, 1/2, ... , 1/2) (as defined by equation (16.11 )) are com- 
putable in polynomial time. This is exactly the aim of the next section. 



72 



Applications to Combinatorial Optimization 



§6.4 



6.4 Derandomization and Arithmetic Circuits 



With respect to the results about derandomization discussed in Subsection [6331 here we inves- 
tigate, for a given NPO problem {R,g, goal) over Z = {0, 1}, whenever it is possible to design 
algorithms to efficiently evaluate Ga having as input some specification of g (and a). To this 
end, we use the notion of arithmetic circuit as a way to specify pseudo-boolean functions (such 
as Ga and g). The semantic of arithmetic circuits, and some relevant properties about them, are 
expressed using polynomials, which are here formally defined. 



6.4.1 Polynomials and arithmetic circuits 



Polynomials 



We begin by recalling some elementary facts about polynomials; for a thorough survey see, for 
example, (lLanglll994l ). Given a ring R we denote with R[xi ,Xn] the (commutative) /?-algebra 
free over {xi ,x„} of polynomials in the indeterminates xi,. . . ,Xn. The degree d&g{p) of a 
polynomial p E R[x\,. . . ,x„] is the maximum degree of its monomials which, in turn, is simply 
the sum of the exponents of the variables appearing in the monomial. Given a /?-algebra 
and a G jz/" we denote with p{a) € jz/ the evaluation of pin a obtained by "replacing" each Xi 
with at in jo; if =c/ = R\y\,. . . ,ym] and qi,...,q„ G R[y\.,- ■ ■ ,ym] we denote with p[qi,. . . ,qn] G 
R[y\ , . . . ,y,„] the substitution of xi, . . . ,Xn with qi,... , qn- We now give some definitions needed 
in the following: 



Definition 6.4.1 Given a polynomial p (zR[x[,. . . ,x„] and a monomial m we denote by [m] [p) 
the coefficient ofm in p (0 ifm is not in p) and with ms^{p) the polynomial obtained by substi- 
tuting every in p with x,-, for each I <i <n and k>0. We denote by /?msf [-^i ) • • • j-^n] ^^e set of 
polynomials in R[xi, . . . ,Xn] whose monomials are square-free. 



It is well known that, in some cases, a polynomial can be uniquely determined given a suitable 
number of its evaluations; in particular, we prove the following: 



Lemma 6.4.1 Any p G /?msf [-fi > ■ • ■ j-^fi] uniquely determined by its evaluations over {0, 1}". 

Proof . Let us introduce some notation to allow a more concise proof. For I <i<n, define e' = 
{e\ ,4) G {0, 1}" such that e) = 1 iff / = j; for every / C {1, . . . ,n}, let e' = Zieie' G {0, 1}" 
and define = (0, . . . , 0). Finally, denote by the monomial Wi^jXi, where denotes the unit 
of R[x\ ,Xn]. It is straightforward to verify that p{e') = T.jci[x^]p = [x']p + T,j'^iW]p> that is 
[x']p = p{e^) — Y,j(^iW]P- Hence, by induction on the cardinality of /, is possible to verify that 
the coefficient of every monomial x' of p is uniquely determined by evaluating p{e^) for / ^ /. 
I 



Arithmetic circuits 



We observe that va rious definitions of arithmetic circu its are present in literature (see, for in- 



stance , the works of lvon zur Gathen and Seroussil (119911) or lValiant. Skyum. Berkowitz. and Rackofi 



(119831) ') and that the differences between those definitions mainly concern the in-degree or out- 
degree. 
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Definition 6.4.2 An arithmetic circuit over the ring R with inputs xi, . . . ,x„ and constants /rom 
S CRis a node-labelled direct acyclic graph such that: nodes with in-degree are labeled with 
elements of SU {xi, . . . ,x„}, nodes with positive in-degree are labeled with elements <?/{+, x} 
and there is a distinguished node, the only one with out-degree 0, called output node. The set of 

all arithmetic circuits over the ring R with inputs xi , . . . ,x„ and constants from S will be denoted 
by 'rf^[xi,. . . ,x„]. The size C(c) of a circuit c is the number of nodes and its depth D{c) is the 
length of the longest path ending in the output node ofc. 

With every node v of an arithmetic circuit over R we recursively associate a polynomial 
p &R[xi,...,Xn] according to its label t 

• if £ G 5U {xi , . . . ,x„}, then p = i; 

• if £ G {+, X }, then p = p\(.p2- ■ -^Pk where pi,...,pk sxe the polynomials associated to 
the predecessors of v. 

We also denote with c the polynomial associated to the output node of a circuit c, so by degree 
(maximum degree) of a circuit c we mean the degree (respectively, maximum degree) of the 
polynomial c. 

Since the main statements of this section are concerned with circuits of size polynomial 
in the number of inputs, it can be easily verified that even when restricting the in-degree and 
out-degree to 2 in the previous definition, none of the presented results would be substantially 
altered. 

Circuits representing polynomials 

In the following we will restrict our attention to the class of circuits 't^^~^'^'^^ [xi,... ,x„] which 
we will simply denote with "^n, using "^^^^ to denote the subset of whose circuits have their 
associated polynomial in Zmst[x\, ■ ■ ■ ,Xn]- These circuits are interesting since they can be ef- 
fectively encoded over a finite alphabet (in size C(c)'^(') if the alphabet is at least binary) and 
then some computations over them can be performed by means of algorithms. We give now an 
example of such a computation: 

Lemma 6.4.2 Given inputs c € 'rf^^^ and a monomial m, [m\c G Z can be computed in 0{C{c)'^d\ogd) 
time, where d = minjdeg (m),n}. 

Proof . If m is not square-free, then [m]c = 0. Otherwise d{m) < n and m can be written as 

Xj' • • -x^" with X\,...,Xn £ {0, 1}: define p = c[Xiz, ■ ■ ■ ,A,nZ] G It is straightforward to 

observe that d{p) = d and [z'^]p = [m]c. The coefficients of are in Z and can be computed 
using the (encoding of) circuit c: the computation requires, for each of the C(c) nodes of c, the 
sum or product of the polynomials associated to its predecessors that are at most C(c). Since all 
the involved polynomials are in Z[z], their sum or product requires at most dlogd steps; given p 
it is then immediate to get the coefficient of z'^, hence the coefficient of m in c. I 

In the following, as we have done in this lemma with a small abuse of notation, we will identify 
a circuit and its encoding. We are interested in circuits of "representing" polynomials; more 
precisely: 
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Definition 6.4.3 A circuit c G ^^[xi ,...,x„] represents p G R[xi, . . . ,Xn] over some subset A of 
a R-algebra whenever c{a) = p{a)for all a G A". A circuit c G 'io^[x\ ,Xn\ exactly represents 
[xi , . . . ,x„] whenever c — p (as polynomials). 

Some interesting facts are known about the representation of polynomials by means of arith- 
metic circuits in %i and about the relationships between circuits representing the same polyno- 
mial: 

Proposition 6.4.1 For every p G Z[xi, . . . ,x„] there exists c exactly representing p (and 
thus representing msf(p) over {0, l}j andc' G "^J"®^ exactly representing msf(p). 

It is clear that each polynomial p (or ms\{p)) immediately "suggests" a circuit; moreover, noting 
that x*^ = X when x G {0, 1} (for k > 1), it is also evident that if c exactly represents p, then c also 
represents msf(/?) over {0, 1}. Since the number of monomials in n indeterminates of maximum 
degree at most 1 is 2", the size of the circuits naively obtained from vr\s^{p) can, in principle, 
be exponential in n. By a simple application of previous observations and of Lemma [6.4. H it is 
simple to notice that: 

Proposition 6.4.2 If c £ "To^^^ represents p G Z[xi, . . . ,x„] over {0, 1}, then c exactly represents 
ms\{p). 



6.4.2 Back to our problem 

Recall that our aim was to investigate when it is possible to design algorithms to efficiently 
evaluate Ga having as input some specification of g (and a), for some NPO problem (/?,g,goal) 
where /? is a /^-relation. 

We have chosen arithmetic circuits as a way to specify pseudo-boolean functions: by this we 
mean that g is specified, for every a G dom(/?), by a family of circuits {c„}„>o with c„ G '^n such 
that, for every (3 G E'" with m = p{\o\), one has that g(a, (3) = Cm(Pi , . . . , Pm)- It is also clear 
from the definition of Ga that this is a polynomial in Zmsf [-^i ,x„], hence it should be specified 
by a family of circuits {c',}„>o with c'„ G '^J"®' such that Ga(pi , . . . , %„) = c'„(Pi , . . . , %„)■ 

Hence, we are interested in the relationship between circuits in %j and in particular 

we would like to find efficient (i.e., polynomial-time) algorithms to transform circuits from one 
class to the other somehow preserving their evaluation. Some similar results are known: for 



instance IValiant et all (119831) have proven 



Proposition 6.4.3 For every c £^nCi circuit c' G %i can be computed in time C(c)'^^^' such that 
d exactly represents c andhas size 0(C{c)^) and depth 0(logC(c)logdeg(c)). 



We now introduce a polynomial that is central to the rest of this note: 

Definition 6.4.4 Let A = (ajj) be anxn matrix with values in {0, 1} and consider the function 
fA{x) = YXi=\Y^]=\aijXj; let Pa G Z[xi,...,x„] be the polynomial whose evaluation coincides 
with fA over Z". 

The polynomial pA has some interesting properties investigated in the next: 
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Lemma 6.4.3 perm(A) = [xi • • •x„]msf(pA) and, for every 5 G N such thatpeYm{A) < 2*, perm(A) = 
2'{2''(""')msf(/7A)(2"', . . . ,2"')} (where {x} is the fractional part ofx G Rj. 

Proof . Since pA = L'^bi ' ' '^ni„Xi^ • ■ -x,-^, where the summation extends over all the n-tuples 
(/i, . ..,/„) G {1, .. . it is easy to conclude that [xi • • ■x„]pA = perm(A), given that the mono- 
mial xi • • - Xn "selects" from the summation only those terms corresponding to tuples that are 
a permutation of {1, . . . ,«}. Then, observing that the only monomial of degree n in vr\s^{pA) is 
xi • • -x,,, it is clear that [xi • • -XnJpA = [-^i • • ■Xn]vr\s^{pA)- Moreover, letting qa = msf (/^a) [z, • • • ,z] S 
Z[z], it is straightforward to verify that ^a(z) = z" perm (A) +^a(2) ^^^^ i'a ^ ^^^^ ^^^^ 
d {q'A ) < " — 1 ■ Hence, for every s such that perm (A ) /2^ < 1 , we have 2"(«-i)^A (2"') = perm (A) /I' + 
2.v(n-i)^^^2^^) thus, since 2-^("^''^^(2^^) G N, perm(A) can be obtained from the fractional part 
of 2-'("-i)gA(2-^'). I 

We begin with an introductory result about algorithms transforming circuits c G ^„ to c' G 
'lo^^^ such that c' represents c over {0, 1}. It is straightforward to observe that: 

Proposition 6.4.4 There exists ca G '^n exactly representing pA of size n'^^^^ and constant depth. 

The circuit ca, which clearly represents pa over {0, 1}, can be obtained from /a in a straightfor- 
ward way. Nonetheless we will show that: 

Theorem 6.4.1 If there exists c G of size n*^^^^ representing pA over {0, 1}, then perm (A) 
can be computed in time n^^^^ having c as input. 

Proof . Since c exactly represents msf(pA) by Proposition 16.4.21 then [m]c = [m]ms\{pA) for 
every monomial m; hence, if C(c) = n^^^\hy lemmas [6.4.2l and l6.4.3[ perm (A) can be computed, 
having c as input, in 0(C(c)^«log«) = n"^'^) time, where n = deg(xi • • -x^). I 

Given that the computation of the permanent is a ftP-complet^ problem ( Valiant . 1979 ). we 
can conclude that: 

Proposition 6.4.5 The problem of mapping a circuit c &'ion to a circuit c' G 'lo^^^ of size n^^^'' 
representing c over {0, 1}, is '^V-hard. 



We now give a stronger result by weakening the hypotheses of Theorem 16.4.11 in the trans- 
formation from c G '^rt to c' G '^n^\ it will now be required that c' represents msf(c) only over 
{1/2}. 

We need introduce a new polynomial, depending on pa. Fix some 5 G N and consider the 
substitution replacing x,- with qi = x,i • • •x,-., for I <i <n; define the function ^,i(xii , . . . ,x„.v) = 
fAiqv (xii • • -xis), . ..qn{x„i ■■■Xns)) and let pA^s G Z[xii, . . . ,x„s] be the polynomial whose eval- 
uation coincides with /a over Z"^. It is straightforward to note that pA[qi, ■ ■ ■ ,qn] = Pa,s and 
that msf(pA,.v) = msf(/7A[^i, • • • ,qn\), since qi and qj have no common indeterminates if / ^ j. In 
analogy with the case of pa, an efficient representation for the polynomial pA,s can be obviously 
deduced from its definition: 

Proposition 6.4.6 There exists ca G exactly representing pa,s of size (ns)'^^^^ and constant 
depth. 

^here, as in jGarev and Johnsorlll979i) . completeness and hardness for jjP are given witfi respect to Turing reduc- 
tions. 
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Nonetheless, having a small circuit representing msf(pA..v) even on only one value can lead to 
the computation of perm (A): 

Theorem 6.4.2 If there exists c £ "^[s^' of size rfi^^^ representing ms\{pj^ „i ) over {1/2}, then 
perm(A) can be computed in n^^^'' time having c as input. 

Proof . By the definition of pA.s it should be clear that msf(/?A)(2^', • • • ,2^^) = 
msf(/)A,i)(l/2, . . . , 1/2) and, by the hypotheses of the theorem, msf(pA,i)(l/2, . . • , 1/2) = c(l/2, . 
and it can be computed, having c as input, in n^^^^i steps. Since a,y € {0, 1}, perm(A) < 2" so, 
by Lemma [6.4.31 perm(A) can be computed taking the fractional part of 2"^("^^V(l/2, . . . , 1 /2) 
and then multiplying by 2" which can be done in overall time n'^^^\ I 
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We are finally able to conclude with the main statement of this chapter: 

Proposition 6.4.7 The problem of mapping a circuit c € %i, exactly representing 
p G 7j[x\ , . . . ,Xn], to a circuit c' £ 'tf^^^ of size n*^^^^ representing msf(p) over {1/2} is '^P-hard. 

6.5 Conclusions 

In this chapter we have discussed an application of uniform random generation to combinato- 
rial optimization. As we have seen, for NPO problems which are RS-good, the existance of a 
polynomial time u.r.g. leads to a polynomial time randomized approximation algorithm with a 
costant performance ratio. The results of this thesis about the existence of polynomial time u.r.g. 
for several classes of p-relations and languages can be apphed to obtain such approximation 
algorithms. 

On the other hand, for NPO problems that are also RS-derandomizable, i.e., for EG prob- 
lems, there also exists the possibility to obtain polynomial time deterministic approximation 
algorithm with a costant performance ratio. In particular, whenever the method of conditional 
expectation can be efficently applied, a "universal" polynomial time approximation algorithm 
can be designed for every problem in EG. 

Nonetheless, in the last section, we have shown that given a NPO problem (/?,g,goal) over 
r = {0,1} where ^ is a pseudo-boolean function specified by means of arithmetic circuit of 
small size, it is in general a (jP-hard problem to obtain a corresponding circuit to evaluate Ga 
(as defined by equation (16.11 ) of Section [6.3.31 ) over {0,1}, or even over 1/2. This negative 
result shows a limit to the derandomization approach to combinatorial optimization and gives 
some evidence that the abovementioned "universal" approach for solving problems in EG cannot 
efficiently solve every such problem (unless FP = jjP). 
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so that there is no sun and no unveiling 

and no host 

only I and then the sheet 
and bulk dead 

Samuel Beckett, Poems in English 



The aim of this thesis has been to investigate, from the structural complexity point of view, 
the relationships between (exact and approximate) counting, ranking and uniform random gen- 
eration and to determine some new classes of combinatorial structures admitting efficient ap- 
proximate counting and uniform random generation. 

We have chosen the PrRAM as & formal model of computation for two main reasons. On 
one side, the PrRAM model is polynomially related to the classical probabilistic Turing machine 
model so that it can be considered, in the usual structural complexity framework, as a feasible 
model of randomized computation. On the other side, the PrRAM showed to be of sufficient 
high level to let us design the algorithms of this thesis avoiding the boaring specifications that 
usually arise when dealing directly with the Turing machine model. 

In describing formally the various problems we dealt with, we have used the notion of p- 
relation, combinatorial structure wA formal language in a somehow decreasing order of gener- 
ality to suggest the broad applicability of the presented results avoiding at the same time unnec- 
essary technicalities in some of the given definitions and proofs. 

To solve the problem of random generation, the first simple idea is related to the ranking of 
formal languages: if we are able to determine a string in a formal language given its position 
(i.e., to solve the unranking problem for that language), the problem of random generation of 
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strings in such a language can be reduced to the random generation of (bounded) integers. We 
have exploited this idea in the general setting of /^-relations, where the unranking problem is 
efficiently solvable whenever the rank can be efficiently computed. We have then extended 
some results about the ranking of formal languages based on particular acceptor devices to the 
case of /7-relations thus obtaining two new classes of /^-relations admitting an efficient solution 
to the uniform random generation problem. 

With regard to this approach, it remains as an open problem to investigate other classes of p- 
relations admitting efficient ranking. For instance, we are now interested in solving the ranking 
pr oblem for languages accepted by automata endowed with both stack and queues, as introduced 



bvlBreveglieri. Cherubini. and Crespi-Reghizzil(ll991l) : lBreveglieri. Cherubini. Citrini. and Crespi-Reghizzi 
(Il996h . used to describe formally, for instance, various operating system scheduling strategies. 



We have also noted that there are cases in which the problem of uniform random generation 
is efficiently solvable when the ranking problem is computationally intractable. This happens, 
for instance, by taking the union of some unambiguous context-free languages. In such cases, 
a very different approach still yields to an efficient solution of the random generation and ap- 
proximate counting problems. This is for instance the case when the combinatorial structure 
whose elements we want to generate uniformly at random can be, in some precise sense, de- 
scribed by means of another structure which in turn admits efficient uniform random generation. 
We have applied such general framework of (ambiguous) descriptions to some classes of formal 
languages, in particular to the case of rational trace languages, languages accepted by poly- 
nomial time nonderministic auxiliary pushdown automata (and hence context-free languages). 
In each case, we have solved in an efficient way the problem of uniform random generation 
and approximate counting whenever the ambiguity of the given language is bounded (finite, or 
polynomial). 

It remains as an open problem to investigate further examples of applications of the (am- 
biguous) descriptions framework. We are in particular interested to approximate solutions for 
difficult counting problems obtained via the use of (ambiguous) descriptions. Some examples 
of the kind of problems we are interested in can be given, fo r instance, by the coeffic i ents o f 



algebraic power series arising in the study of formal languages IChoffrut and GoldwurmI (119951) . 



Finally, we have applied our results about random generation to an heuristic for improving 
local search algorithms used in combinatorial optimization. This application is in particular 
motivated from the fact that we have proved that the derandomization of such an heuristic can, 
in some case, be a jjP-hard problem. 

With regard to the latter issue, it remains as an open problem to relate the formal results 
obtained here in terms of acceptor devices to various examples of natural combinatorial opti- 
mization problems as presented, for instance, in lCrescenzi and KannI (Il998h . 
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It was as if these depths constantly bridged over by a structure that was firm 
enought in spite of its lightness and of its occasional oscillation in the somewhat 
vertiginous air, invited on occasion, in the interest of their nerves, a dropping of the 
plummet and a measurement of the abyss. 

Henry James, The Beast in the Jungle 



The aim of this appendix is to separately discuss some technical and probabilistic aspects of 
this thesis in order not to crowd with details its main part. 

A.l Some Lemmas 

In this section we give the proof of some technical results that are needed in the thesis. We start 
by an inequality we have often used in proving several upper bounds: 

Lemma A. 1.1 For every a > and P > 0, there exists a constant K > such that, for every 
< e < 1/(3 and 5 > 0, it holds that 

a(l-(3e)t'°^{i'i°g(i/5)} <5_ 

Proof . By taking the logs, the above inequality becomes Kmax{l,log(l/6)}/slog(l — Ps) < 
log(6/a). Since log(l — x) < —x, for x < 1, the inequality holds whenever 

-Kpmax{l,log(l/5)} < log(5/a). 

Therefore, if K > (1 +max{0,loga})/P > 0, we have 

1 / log(l/§) max{0,loga} \ 

^ P Vmax{l,log(l/6)}^max{l,log(l/5)}y' - 
^ 1 log(l/5) + loga ^ log(5/a) 
- pmax{l,log(l/5)} pmax{l,log(l/6)} ' 
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Now we consider two algorithms that, respectively, compute the least common multiple 
(1cm) of the first n integers and the integer part of the logarithm of n: 

Lemma A. 1.2 The least common multiple flcmj o/ {1, . . . ,«} is an integer of 0{n) bits and it 
can be computed in 0{n^) time by a RAM under logarithmic cost criterion. 



Proof . First, we recall that lcm{l, ...,«} < n'^^"', where 7l(?i) is the ?i-th prime dSzepietowskil . 



Lemma 4.1.2.). As a consequence, lcm{l, . . . ,«} has 0{n) bits since it is well known 



that 7i(m) ^ n/logn. Moreover, a naive iterative procedure, based on Euclid's algorithm for 
computing the g.c.d. can be designed to compute lcm{l, ... ,n} in 0{n^) time. I 



Lemma A. 1.3 The number [logA'^] of bits required to represent {1, . . . ,A'^} can be computed in 
0{\ogN\og\ogN) by a RAM under logarithmic cost criterion. 

Proof . We compute [logA'^] by the call Size(A'^ — 1), where "Size" is the recursive procedure 
given by Algorithm lA.il 



procedure Size(?i) 
if « < 1 then return 1 
else 

h -(^ \, ho h 

k 2, Icq k 

while k <ndo 

ho ^ h,h -i^ h + h 
ko k,k ^ k-k 

return /jo+Size(?2/^o)- 



Algorithm A.l: Computing the bit size of an integer. 

A simple analysis shows that the time complexity of Size(?i) is given by C?(Lf^i iT) 
0{kl'^), where k = log log «. I 



A.l Probabilisitc Detour 

This section is intended to give a separate discussion of the probabilistic aspects involved in 
the thesis. In this way, we hope to give a more precise and detailed evidence of some of the 
properties used in order to prove the correctness of algorithms we have presented in the previous 
chapters. 



A.2.1 Hoeffding's inequality 

Hoeffding 's inequality dHoeffdind J 1963h is the main tool used in this work to prove concentra- 
tion results, i.e., to show that the sum of certain random variables has, with high probability, a 
value very near to its expec tation. This inequality is based on the so called Chemoff's bounding 
technique dChemoffl. 119521) that uses an exponential version of Markov 's inequality. Here, we 
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want to give a concise proof of such an inequality since is based on a very simple and intuitive 
idea we would like to elucidate. 



Let {Q., ^ ,P) be a probability space, and consider a random variable X on it, i.e., a measur- 
able function X : H — )■ R. The expectation of X, denoted by E[Z], is defined as 



XdP 



where the integra l is understood in the Lebesgue sense (for a detailed discussion of these details, 
see, for instance (IBillingsleyl . Il979l : lFellerlll968h ). If, for every CO € il, it holds that X((o) > 0, 
then, for every t > 0, 

[ XdP > f tdP+ f XdP > tP{X >t} + 0. 
■la Jx>t Jx<t 

Hence, we have proven Markov's inequality: 



Now, by taking the exponential, we can apply this inequality for every random variable Y, 
even those assuming negative values: for every a > 0, we have 



E[e 



aY] 



Chemoff 's bounding idea consists in chooseing a such that the right hand side of the previous 
inequality is minimized. We can apply the inequality to a sum of independent random variables 
Xi,... ,X„: 



P{{txi-E[txi])>t}<e-'E 
^ i=i i=i ^ 



exp(a£(X,--E[X;])) 

!=1 



(A.l) 



i=\ 



,a(X,-E[Z,]) 



where the equality follows from the independence of the X/s. In order to choose the right a, 
we need an upper bound for E[e"^] (where, in this case, F = X,- — E[X,]). Assume that Y is 
bounded, i.e., that there exist some & G R such that P{a <Y <b} = \; then, by convexity of 
the exponential function. 



,a(y-E[r]) 



< 



^ah _ g(u) 



where u = a{b — a) and g{u) = —pu + log(l — p + pe") with p = —xj (h — a). Is easy to verify 
that g{0) = g'{0) = and that g"{u) < 1 /4. Hence by Taylor's expansion, for a suitable 6 € R, 

giu)=g{0) + ug'iO) + '^g"{Q)<'^ 
and we are able to conclude that 



E[, 



< e 



aE[Y]+a-{h-af/S 
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We are ready to give a proof of Hoeffding's inequality. Let Xi,. . . ,X„ be independent ran- 
dom variables such that P{X,- G [a,-,Z>,]} = 1 for 1 < / < n; then, for every £ > 0, by applying 
equation (lA.ll ) and taking a = 4£/^''^i {bi — ai)^,we have 



We conclude this section by stating two lemmas that restate Hoeffding's inequality in a form 
which is used in some of the proofs of this thesis. Here, we use the notation Pr{A} to denote 
the probability of an event A whenever we do not want to formally specify the probability space 
over which the event itself is defined. 

Lemma A.2.1 Let X,Xi ,X„ be independent, identically distributed, binomial random vari- 
ables such that Pr{X = 1} = 1/2 + 5, /or 1/2 < 6 < 1. Then, 



Proof . The statement follows by taking bi — a, = 1, for 1 < / < n, and replacing £ by nh. I 

Lemma A. 2.2 Let X,Xi,...,X„ be independent and identically distributed random variables 
taking values in [0, 1]. Then, for every £ > 0, 



Proof . The claim follows by taking bi — a,- = 1, for 1 < / < n, and replacing £ by mE{X). I 
A.2.2 Improving randomized algorithms 

As we have seen, randomized algorithms sometimes fail to give the correct answer and somehow 
signal this fact, let us say by giving an "undefined" output denoted by the special symbol _L. A 
common strategy to decrease the probability of such an event, is to iterate the algorithm, usually 
for no more than a specified number of times, and to give as output the first non-undefined 
output, or some other function of the non-undefined outputs obtained in the iterations (or to 
output ± if, for all the specified number of iterations, the algorithm always outputs _L). 

Here, we try to formalize this idea only with regard to probabilisty issues. For this reason, 
all the interpretations of the probabilistic statements of this subsection in terms of randomized 
algorithms is printed in a smaller font. 

Let X,Xi, . . . ,X„ be independent and identically distributed random variables taking values 
in a set X C R and let A' C X be a fixed measurable set. 




In a similar way, we can obtain that 
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Clearly, our idea is to use X to represent the output of a randomized algorithm and K to mean 
the set of its intended output. Usually, in this thesis, K = N and X is as simple as {-L}, 
where we assume to represent ± in R by some number not in K. Finally, the X,'s represent the 
output of n iterations of the algorithm. 

Let now F, Fi , . . . , y„ be independent and identically distributed random variables taking val- 
ues in X such that 

Pr{F eH} = Pr{X eH\X£K} (A.2) 
for every measurable set H C K. 

Again, our idea is to use Y to represent the intended output of the algorithm, together with 
its desired distribution. Equation ( IA.2I ). in fact, formally states that Y should be distributed 
like X whenever (conditionally to) X is different from _L. The same, due to the identity of the 
distribution, is true for the y,'s. 



We now show that, for every measurable function F : K" —^^ R™ and every measurable set 
M G R™, it holds that 

Pr{F{Xy,...,X„)eM\XiGK,...,X,£K} = Fr{F{Yu...,Yn)eM}. (A.3) 

For the sake of brevity, let 

Ff^M) = {x : F{yi, . . . ,yi^i,x,yi+i, . . . ,yn) e M,yi, . . . ,yn £ K} 

for I <i <n. Then 

Pr{nLi Xi G F.-^{M) n nLi Xi G K} 



Pr{F(Xi, . ..,Xn)eM\XieK,... ,Xn G K] 



Pr{X G K]"' 
^ Pv{Xi£Fr\M)r\XieK} 
/=i Pr{^ e K} 

n 

Y\Pv{X £Fr\M) \ X(^K} 

i=\ 

f\Pr{Y eFr\M)] 



i=\ 



Pv{{^Yi£Fr\M)] 

i=\ 

Pr{F(Fi,...,F„) GM} 



Equation ( IA.3b states that the probability of every event F{Xi,. .. ,X„) G M obtained taking 
some function F of the outputs of n iterations of a probabilistic algorithm, given that all the 
iterations are non-undefined, is equal to the probability of the same event F{Yi,... ,Y„) G M, 
expressed in terms of the intended outputs distribution. 
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Elementary case 



Here we investigate the first strategy mentioned at the beginning of this section: to iterate the 
algorithm, giving in output the first non-undefined output so obtained. 



Under the previous hypotheses, let Tk = mf{t . I < t < n,Xt G K} (assuming, as usual, 
inf = oo). Then, for every measurable set H C K,it holds that 

(i) Pt{X^^ g // I < 00} = Pr{X (£H\X£K} = Pr{F G H}, 

(ii) Pt{1k = 00} = Pr{X K}". 

The second property is trivially verifiable; the first comes from 

PT{x,,eHn[ji=i^K = i} 



Fr{X^, G // I Xjf < 00} 



V= 1 Pr{np\ Xj^KnXjGKnXjGH} 

LU^myJiXj^KnXieK} 

Pr{XeH}l^UFv{n%\Xj^K} 
Pr{XGK}l^UP^{n%\Xj^K} 
Pr{X eH \ X eK}. 



Property ^ tells that if one of the iterations of the algorithm is ever successful, then that output 
has the same distribution as the intended one. Moreover, Property ^ says that the probability 
that such "iteration strategy" has an undefined output decreases exponentially with the number 
of iterations. 



A more complex situation 

Now we assume that, after a certain number of iterations of a randomized algorithm, we want 
to return some function of the outputs of all the non-undefined iterations, such as, for instance, 
their median, or mean. 



Under the previous hypotheses, given a family of R valued measurable functions {/n}n>o. 
let {f^}n>o be the corresponding family of functions defined as 

fn {-^l: ■ ■ ■ ■>-^n) — fm (-^I'l ) • • • i ) 

where ii < ■ ■ ■ < are. exactly all the indexes for which x,„, G K with 1 < k < m < n, for every 
n>0. Observe that the functions /,f are obviously all R valued measurable functions, for « > 0. 



Clearly, /,f is the function computed at the end of the n iterations on all the outputs, that, 
by definition, computes the function (for instance, the mean, or the median) only on the k 
non-undefined outputs of all the iterations. 
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Let r\K = \ {t '■ 1 <t <n,Xt G ^}|. Then for every measurable set // C ^ and every ^ > 0, it 
holds that 

Pr{/,f (Xi, . . . ,X„) eH\^K = k} = Pr{A(Xi, . . . ,Xt) e H \ e K, . . . ,Xt e K} 

= Pr{MYy,...,Yk)eH}. 



This property formally states that the above mentioned strategy is "correct", in the sense that 
the value of /,f on all outputs, given there were k non-undefined outputs, is distributed as f/^ 
computed over k intended outputs of the algorithm. 

The second equality comes from equation (IA.3I) . To prove the first inequality, define, for the 
sake of brevity, the events 

A{r) = f]XieK e B(r)= f] Xt^K- 

i&T i<i<nJ<^r 

where F = (/i , . . . , ik) is any ordered ^-tuple of indexes without repetitions, i.e., 1 < /i < • ■ • < 
4 < n and I <k <n. Then, 

Pr{/„ (Xi , . . . ,X„) eH\^,-k}- ('^)Pr{XE/^}^Pr{X^^}-^ 

_ LrHh...J,)Mfk{Xi,,...,Xi^)eHnA{r)}Fr{B{r)} 

Q Pr{X e i^}*^Pr{X ^ K}"-'' 
_ {DPr{MXu...,Xt)eHnA{{l,...,k})}Fr{X<^K}"-' 
(l) Pr{X € 7S:}^Pr{Z K}"-'' 

^ PT{MXu...,Xt)eHnA{{i,...,k})} 

Pr{X G KY 
= Pr{A(Xi, . ..,Xk)eH\XiGK,...,Xke K}. 



An upper bound 

Usually, we would like to prove some concentration result about the output obtained applying 
ff. To this aim, we assume to have an (exponential) upper bound on the probability of the 
concentration of applied to k intended outputs of the algorithm. Hence, we show a corre- 
sponding (exponential) bound on the concentration of f^. 

Under the previous hypotheses, define sk = Pr{X € K} and, for a given measurable set 
H QK, assume that there exists a < c// < 1 such that, for every ^ > 0, 

Pr{MYi,...,Yt) eH}<e-^'^. 

Then, it holds that 

Pr{/,f (Xi, . . . ,X„) e // I nj, > 0} < - (l - . (A.4) 
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Observe, in fact, that 

Fr|/„ [Xi,... ,X„) en \r\K>0^ PrjTlj^ > 0} 

^ LLi PrUf (^b ■ • . e I rii^ = k}Pr{^K = k} 
l-{\-Pr{X eK})" 

l-(l-Pr{X ei^})" 

ILo ('^) {e-'"Pr{x e /:})^Pr{x ^ ji:}"-'- 

l-{l-Fr{X gK})" 
(l-Pr{X£^}(l-g-""))" 

l-(l-Pr{X e/^})" 
(l-Pr{X£/:}c^/2)" 
l-(l-Pr{XGii:}) 



< 



< 



< 



where the last inequality follows from the facts that l—e ^ >x/2 and x" <x whenever x E [0, 1] 
and « > 0. 



A.2.3 The method of conditional expectation 



We conclude this appendix with a detailed discussion, from the probabilistic viewpoint, of the so 
called method of conditional expectation. Let Xi, . . . ,X„ be independent random variables (not 
necessarily identically distributed); moreover, to avoid boring technical details concerning the 
existence of the conditional expectation used in the following, assume that each X,- takes values 
in some discrete sefl-X^ C R, for I <i <n. Finally, let / : R" ^ R be some fixed function. 

If we define Z as the random variable Z((o) = f{X\ (co), . . . ,X„((o)), then, by the internality 
of the expectation, 

inf Z((o) < E[Z] < supZ((o). 

men 

In our hypotheses, this means in particular that there exists some & gQ. such that E[Z] < Z{G)), 
or, that there exist Xi = X,(a)) S R, for I <i <n, such that 

E[/(Zi,...,X„)]</(xi,...,x„). 

The aim of the method of conditional expectation is to obtain such x,'s by computing some 
conditional expectations. 

For the sake of brevity define, for I < k < n, the functions : R'^ — R such that 

gk{xi,...,Xk) =E[Z |Xi =xi,...,Xk=Xk]. 

By a straightforward application of the definition of conditional expectation and the elementary 
property of iterated expectations, one can verify that the following statements are true, for every 
xi,. . . ,x„ € R and 1 <k<n: 



(a) E[Z]=E[gi(Xi)], 

(b) gkixi,...,Xk) =E[gk+iixi,...,Xk,Xk+i)] and 

'this is usually stated by sayng that the X,'s are simple random variables. 
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(C) gn{xi,...,Xn) = f{xi,...,X„), 

(d) E[Z] < msix_,-exg\(.x) and 

(e) gk{xi,...,Xk) <maxxexgk+i{x\,.-.,xk,x). 

Hence, if the x, 's are recursively defined as 

^ ^ fargmax^gxgi(-^) if ' = L 

[argmax^gX^K^i > • • • ^-^/-i if ^ > 

from ©-dell, one can conclude that E[Z] < /(xi , . . . ,x„). 



Symbols and Abbreviations 



In this appendix we summarize some notaion, together with some pointer to relevant literature 
for the notions used in this thesis that are not explicitly defined in other chapters. 



Some Complexity and Approximation Classes 



langu ages recognizable in polynomial time by a (deterministic) Turing machine (iGarey and Johnsonl . 
1979) . 



FP 



NP 



funct ions computable in polynomial time by a (deterministic) Turing machine ( Garey and Johnsonl. 
1979I) . 



langu ages recognizable in polynomial time by a nondeterministic Turing machine (IGarey and Johnson 
197^ . 



functions yelding the number of acce pting computations in nondeterministic polynomial 
time Turing machines (IValiantlll979l) . 



RP langu ages recongizable in probabilistic polynomial time with one-sided error (IB alcazar et all 



1995h . 



languages recognized (functions computed by) log-space uniform boolean circuits of bounded 

fan-in , polynomial size and 0{log^n) depth ( Karp and Ramachandran . 1990 ; B alcazar. Diaz, and GabarroL 
1990l) . 



(I^{s{n),i{n),d{n))) class of languages recognized by Turing machines with simultaneous 
complexity bounds (Section [ 



NPO 
GLO 
EG 

APX 
PTAS 
FPTAS 
MaxNP 



NP optimization problems (Section [6TT]) . 

NPO problems with guaranteed local optima (Section r6.2.2l ). 

(and RS-good, RS-derandomizable) expectation-guaranteed NPO problems and related 
subclasses (Section [ 



NPO problems approximable in polynomial time within a costant factor (Section [6. 1.2I ). 
NPO problems admitting a polynomial time approximation scheme (Section [6. 1.2I ). 
NPO problems admitting a fully polynomial time approximation scheme (Section [6. 1.2I ). 
(and MaxSNP) logically defined NPO problems (Section Igni ). 
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Models of Computation 

RAM random access machine (Section fl-l-lb . 

PrRAM probabilistic RAM (Section [TTlll 

TM (NTM) (nondeterministic) Turing machine (Section 

1-AuxPDA (1-NAuxPDA, 1-UAuxPDA) (nondeterministic, unambibuous) one-way auxihary push- 
down automata (Section [ 



Number Rings and Fields 

N, Z, Q and R denote, respectively, the ring of integer and natural number and the field of 
rational and real numbers. 



Basic notation 

I CO I the lenght of the string (O 
#A the cardinality of set A 



List of Algorithms 



1.1 a uniform random integer numbers generator. 

3.1 An algorithm to increase the success probability of a u.r.g 

3.2 An algorithm to increase the probability of correct approximation of a r.a.s.. 

3.3 a uniform random generator for regular languages 

3.4 An u.r.g. for (ambiguously) described combinatorial structures 

3.5 A r.a.s. for ambiguously described combinatorial structures 

3.6 A u.r.g. for the product of combinatorial structures 

3.7 A u.r.g. for the union of combinatorial structures 

4.1 a uniform random generator of derivation trees 

4.2 Part of Barley's algorithm modified to count derivation trees 

4.3 Counting derivation trees 

6.1 Local Search: schema of algorithm 

6.2 RandomSearch 

6.3 Conditional expectation 

6.4 An approximation algorithm for EG problems 



A. 1 Computing the bit size of an integer. 
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extended, 25 
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extended, 21 
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